I've made that point before. From an "alignment" perspective, AIs and corporations have many of the same problems. Corporations, according to Milton Friedman, have the sole duty of increasing their profits.[1] This is a key document in the history of modern conservatism.[2]
So what happens when AIs are "aligned" purely to corporate goals? This is a nearer term worry than machines taking over for the benefit of machines. AIs working hard for the benefit of corporations are already here. Try any customer service chatbot.
There's reluctance to think about this too hard. It raises embarrassing questions about the role of the corporation in society. It can become a challenge to corporate personhood.
That's threatening to many people in power. Dealing with it may be unavoidable as AIs take a stronger role in managing companies.
Also the accountability problem! Sometimes we try to hold the corporation accountable and sometimes the individuals. It never seems adequate though.
I found the documentary "The Corporation" (2003) very instructive in that regard, particularly the segment with the BP CEO where the point is made that a company made up ethical individuals can still act unethically, or unaligned to human values, just by virtue of the incentive and legal structure governing the corporation.
