> LeCun isn't saying that LLMs aren't useful. He's just concerned with bigger problems, like AGI, which he believes cannot be solved purely through linguistic analysis.
It feels like special pleading: surely _this_ will be the problem class that doesn’t fall to “the bitter lesson”.
My intuition is that the main problem with the current architecture is that mapping into tokens causes quantization that a real brain doesn’t have, and lack of plasticity.
I don’t build models, I spend 100% of my time reading and adjusting model outputs though.
It feels like special pleading: surely _this_ will be the problem class that doesn’t fall to “the bitter lesson”.
My intuition is that the main problem with the current architecture is that mapping into tokens causes quantization that a real brain doesn’t have, and lack of plasticity.
I don’t build models, I spend 100% of my time reading and adjusting model outputs though.