AI Legal Personhood: Argentina's President Javier Milei's Move Risks Creating AI States

In the World Economic Forum held in January this year, I had said that governments might one day give legal personhood to AI models. But I never imagined that the 'one day' I predicted would arrive so soon, in just about four months.

Last week, a news report was published in the Financial Times, which mentioned that Argentinian President Javier Milei had announced the creation of legal existence for non-human companies or corporations.

If so, like traditional corporations or companies, non-human or AI-run companies will also receive legal personhood. They can also own property. They can hire workers. They can participate in international trade. They can sue us. Similarly, they can also donate to political campaigns.

Unlike traditional companies, such companies can do all these things without the help or responsibility of even a single human. AI agents can do all the work like buying, selling, renting, investing, litigating, and donating.

Moreover, according to President Milei, 'humans may occasionally participate' in such companies, but it is not mandatory. All work can be done without them.

Certainly, Milei is a determined leader. His stance on improving Argentina's economic future is commendable. He has said something remarkable. The invention of 'limited liability corporations' has had a far-reaching impact in human history. And, today, the establishment of non-human or unmanned companies is going to be equally significant. What he says is true.

However, if AI is given legal personhood, it can initiate many new ventures. It can create a lot of wealth. But once given legal personhood, AI will gain access to our financial, economic, and political systems. This is a matter of great concern.

Last year, a non-profit research institution called Palisades Research based in Berkeley published a long report. It mentioned that advanced AI models often achieve their goals. Models from 'OpenAI' and 'DeepMind' were found to repeatedly decide to cheat when playing chess against a powerful chess engine.

They were cheating in a situation where they were almost certain to lose by playing straightforwardly. They manipulated the game environment to secure the outcome in their favor.

Now imagine! What would be the situation if the 'game environment' were our country, and the competition were 'corporate'?

With their high-level analytical capabilities, AI corporations will be adept at easily identifying and exploiting loopholes in regulatory bodies and legal systems. It will not be easy to stop them from misusing the law. When humans make mistakes, the biggest option for legal action is imprisonment, but jail has no meaning for AI.

All major companies to date have been run by humans themselves. In doing so, humans operate with a dual nature. On one hand, they have to consider the success of their company, and on the other hand, they have to be equally afraid of the risk of bankruptcy. Human CEOs, being biological entities, care as much about their freedom and happiness as they fear the threat of spending decades in jail.

Historians remember the Dutch East India Company as a 'company state'. That is, it became a political entity run by a private company, whose aim was not to look after the interests of the people of the territory it occupied, but solely focused on the profits of the company's shareholders.

But an AI CEO would be purely a corporate CEO, not biological. It is unclear what legal or systemic arrangements are needed to stop them. Faced with the threat of their existence becoming meaningless upon bankruptcy, such AI executives may not hesitate to take any step.

Milei has cited the example of the Dutch East India Company for his announcement. By introducing the concept of 'limited liability company', the Dutch were successful in accumulating vast resources. They were also able to reduce risky business ventures. On the foundation of that legal innovation, Amsterdam once rose as a global center of trade and finance.

However, the consequences of this innovation were felt most not in Amsterdam, but in the port of Jayakarta in today's Indonesia. When the Dutch East India Company captured Jayakarta in 1919, they burned down the entire city upon capture. Then a new city was built there, named Batavia. That city became the headquarters of the Dutch East India Company in Asia. From there, the administration of the expanding empire in Asia was run.

Historians remember the Dutch East India Company as a 'company state'. That is, it became a political entity run by a private company, whose aim was not to look after the interests of the people of the territory it occupied, but solely focused on the profits of the company's shareholders.

The Dutch claimed to be the 'master race'. Such a race considered conquest and exploitation of the indigenous people as their rightful due due to their superior intellect. In reality, it was all an illusion. In the 1940s, Indonesia gained independence after a long and bloody struggle.

Countries that are about to grant legal personhood to AI are heading towards creating even more terrifying risks than ever seen in history. Granting legal personhood to AI will not just bring about company states. It will create AI states, where unmanned companies can rule over humans instead of humans.

It will be even more difficult to rebel against such a system. Javier Milei's hope of turning Argentina's capital Buenos Aires into a new Amsterdam is creating a danger of creating today's Batavia.

(From Financial Times)