My experience with asking ChatGPT to write code is that is produces code that LOOKS like it will work and solve the question asked but it actually doesn't. For example, I've asked it to create code examples of how to use different features in some Python libraries. The samples it produces make me think "ok, that's exactly how I would expect X feature in this library to work", but upon more a detailed inspection, I find that it references library methods that don't even exist! You can complain to the bot that it is lying and it will apologize and spit out another sample with calls to nonexistent methods.
This experience makes me question if something else is going on here. It's easy to overlook a mistake in a whiteboard coding exercise.
I think of it as the model having lossy compression. The library it writes the feature against is the one you expect to exist, based on your overall understanding of the problem, not the one that actually exists. It has learned the low resolution version of the problem, not the specific library it is referring to.
I've asked it to make React components, with unit tests, etc and it works quite well. Also things like image to ASCII converter in a webpage, adding dithering, etc. Found only tiny bugs here and there.
The thing is if it were a static language this state is just 1 step away from full-blown programs, as it could see the function does not exist, and ask it to generate the body.
This experience makes me question if something else is going on here. It's easy to overlook a mistake in a whiteboard coding exercise.