UNIX isn't getting automated (or at least not well). I've been using it for decades and the usability has hardly improved. UNIX is actually being replaced by cloud vendors, as the AWS/Azure/GCP/OCI API isn't UNIX. That's kinda the point - I was thinking of automating it a bit more, but it turns out my friend doesn't need better automation because why "automate" something in the conventional sense when you can just throw AI bodies at doing all the work manually.
The risk here is not that AI can't explain what a particular script does. The risk is that we stop creating simplifying abstractions. Let me try to explain what I'm getting at here because it's a new thought and I'm not doing it well.
Humans have limited and roughly equal mental and physical capacity. Beyond a certain point you have to add more people to get more done. In turn those people need abstractions because they can't grok the whole system (AI assistance or not), and so to organize into teams you need people to abstract away essential complexity. We build on top of a tower of abstractions as a consequence, and to a large degree progress comes via better and higher level abstractions becoming available.
What if this progress stops because AI doesn't need it? The scenario here is that without people really noticing at first the flow of improvements to our tools dry up. PEPs, JEPs, RFCs, new libraries etc stop appearing. Maybe even bugs stop getting fixed if there are workarounds available. After all, why invest time and effort into improving the syntax of Python or JavaScript if the primary user would only be AIs? They don't care, they'll just write things out the inconvenient way without complaining. Why try to make UNIX or the cloud easier to use, if all your configs are written by AI? Why introduce new abstractions at all?
Assume that tech continues to move forward and people's demands continue to get more complex. But now, instead of inventing new tools, we just pay a bit more for AI to brute force through the limitations of old tools. Why would an AI write and maintain a library on GitHub with a nice API when it can just knock off the code it happens to need right there as part of its own task? So code gets more and more verbose over time, until one day the number of missing abstractions becomes so large that the code is incomprehensible. And asking AI to explain won't help either because at best you'd get a book sized explanation. There are no teams, no projects, no abstractions. It's all just an endless jungle of boilerplate and copy/paste coding.
This is an interesting thought and I have to agree it’s a legitimate issue. I have also considered something similar - that in the near future, if all software is designed with AI, or “generated” rather than explicitly engineered, we would certainly reap some benefits from that but we would also cease understanding how anything actually works. Obviously no one person fully understands the abstractions we have currently engineered when considering the totality.. but at least currently we have them.
The risk here is not that AI can't explain what a particular script does. The risk is that we stop creating simplifying abstractions. Let me try to explain what I'm getting at here because it's a new thought and I'm not doing it well.
Humans have limited and roughly equal mental and physical capacity. Beyond a certain point you have to add more people to get more done. In turn those people need abstractions because they can't grok the whole system (AI assistance or not), and so to organize into teams you need people to abstract away essential complexity. We build on top of a tower of abstractions as a consequence, and to a large degree progress comes via better and higher level abstractions becoming available.
What if this progress stops because AI doesn't need it? The scenario here is that without people really noticing at first the flow of improvements to our tools dry up. PEPs, JEPs, RFCs, new libraries etc stop appearing. Maybe even bugs stop getting fixed if there are workarounds available. After all, why invest time and effort into improving the syntax of Python or JavaScript if the primary user would only be AIs? They don't care, they'll just write things out the inconvenient way without complaining. Why try to make UNIX or the cloud easier to use, if all your configs are written by AI? Why introduce new abstractions at all?
Assume that tech continues to move forward and people's demands continue to get more complex. But now, instead of inventing new tools, we just pay a bit more for AI to brute force through the limitations of old tools. Why would an AI write and maintain a library on GitHub with a nice API when it can just knock off the code it happens to need right there as part of its own task? So code gets more and more verbose over time, until one day the number of missing abstractions becomes so large that the code is incomprehensible. And asking AI to explain won't help either because at best you'd get a book sized explanation. There are no teams, no projects, no abstractions. It's all just an endless jungle of boilerplate and copy/paste coding.