I've been calling this out since the rise of ChatGPT:
"The real danger lies in their seductive nature - over how tempting it becomes to immediately reach for the LLM to provide an answer, rather than taking a few moments to quietly ponder the problem on your own. By reaching for it to solve any problem at nearly an instinctual level you are completely failing to cultivate an intrinsically valuable skill - that of critical reasoning."
I've had multiple situations where AI has helped me get to the solution because it has been unable to get there itself. But that I wouldn't have realised that solution otherwise. In one case, looking for a plot, it delivered many woeful options but one sparked an alternative thought that got me on track. In other cases trying to debug code, having it talk through the logic/flow and exhaust other fixes, I have managed to solve the problem despite not being experienced at all with that language.
The dangers I've found personally are more around how it eases busywork, so I'm more inclined to be distracted doing that as though it delivers actual progress.
I agree in principal - the process of problem solving is the important part.
However I think LLMs make you do more of this because of what you can offload to the LLM. You can offload the simpler things. But for the complex questions that cut across multiple domains and have a lot of ambiguity? You're still going to have to sit down and think about it. Maybe once you've broken it into sufficiently smaller problems you can use the LLM.
If we're worried about abstract problem solving skills that doesnt really go away with better tools. It goes away when we arent the ones using the tools.
You can offload the simpler things, but struggling with the simpler things is how you build the skills to handle the more complex ones that you can't hand off.
If the simpler thing in question is a task you've already mastered, then you're not losing much by asking an LLM to help you with it. If it's not trivial to you though, then you're missing an opportunity to learn.
The biology of the human brain will not change as a result of these LLMs. We are imperfect and will tend to take the easiest route in most cases. Having an "all powerful" tool that can offload the important work of figuring out tough problems seems like it will lead to a society less capable in solving complex problems.
"The real danger lies in their seductive nature - over how tempting it becomes to immediately reach for the LLM to provide an answer, rather than taking a few moments to quietly ponder the problem on your own. By reaching for it to solve any problem at nearly an instinctual level you are completely failing to cultivate an intrinsically valuable skill - that of critical reasoning."