What does asymmetric key agreement have to do with using ChaCha?
There are definitely ways in which GCM is more brittle than ChaCha; in particular, you can use random nonces with XChaCha, and you technically can't safely do that with GCM. But really, throw a dart, your outcome will be the same either way.
I personally prefer the SIV modes, but using a sequential nonce within a session is pretty much always better due to birthday attack stuff.
Cryptography does advance over time. When GCM was introduced it was when everyone realized that you must always have authentication and they wanted a faster auth mode than HMAC.
Makes me wonder what new understandings might be coming down the road other than the obvious quantum computing factor. The most recent one I remember is the risk of combining compression with crypto if the attacker can influence anything in the compressed data.
There are definitely ways in which GCM is more brittle than ChaCha; in particular, you can use random nonces with XChaCha, and you technically can't safely do that with GCM. But really, throw a dart, your outcome will be the same either way.