AI aftermath scenarios: eight futures people debate

AI aftermath scenarios are stories about the world after machines can outperform people in almost every domain. They ask what that world looks like and whether it is a utopia or a dystopia. Wikipedia's article on AI aftermath scenarios sorts them into eight kinds, from peaceful coexistence to human extinction. This post walks through each one in plain words, says what it assumes, and ends with a way to compare them. They are thought experiments, not forecasts.

AI aftermath scenarios: the two ideas they rest on

The article starts from a belief held by most scientists it describes: AI research will at some point produce machines as intelligent as people, or more so, in every domain of interest. It gives a physical argument. No physical law stops particles from being arranged to compute even better than a human brain, so superintelligence is physically possible. A digital brain could also be far larger and faster, since human brains had to fit through a birth canal.

There is no agreement on when this might happen. Many scholars say a second species of intelligent life on the planet would have far-reaching effects, yet they disagree about which scenario is most likely and which is most desirable.

Every scenario below depends on two linked ideas:

• A post-scarcity economy. Economic growth continues until goods are so cheap that a comfortable life for everyone no longer needs most people to work, unless the wealth is extremely concentrated.
• Humans become unnecessary to the economy. Human labor loses value if robots are cheaper to mass-produce, easier to customize and more capable than people.

If you want the word behind the first step, read artificial superintelligence (ASI): what it is and is not.

Two economic futures: libertarian and communist

In the libertarian scenarios, intelligent machines, uploaded humans, cyborgs and unenhanced humans live side by side under respect for property rights. Because machines make goods without scarce human labor, land becomes very valuable. Even a person who stays "unenhanced" could lease a small piece of land to far more productive robots and live on the payments.

The article lists the weak point right away. Such a decentralized world may be unstable. The greediest superintelligent members would have both the means and the motive to take the property of the unenhanced, even if the law has no loopholes, simply by tricking them into signing it over. Suffering could be widespread, because beings with no property could die.

The communist scenario comes from Ray Kurzweil, who argues that technology could realize the maxim "from each according to his ability, to each according to his needs." The route is not revolution. Low manufacturing costs, abundant materials and open-source design would remove scarcity itself. For his wider argument, see The Singularity Is Near by Ray Kurzweil: the main ideas.

The article names the catch. The transition could create new inequality between people who control AI systems and people who do not, and it is unclear how to give everyone fair access. A small elite holding the AI could mean a new kind of class division.

The benevolent dictator

In this scenario a superintelligent AI takes control of society, but for good ends. Its programmers, working on a deadline, somehow solved problems that had seemed hopeless, and gave it one goal: work out what a human utopia looks like by studying human behavior, brains and genes, then build it.

The AI values diversity, so it splits Earth into sectors. Harming others, making weapons, evading surveillance and building a rival superintelligence are banned everywhere; otherwise each sector sets its own laws. Disease, poverty, crime and involuntary suffering are gone. The article describes this scenario as it appears in Max Tegmark's book, covered in Life 3.0 by Max Tegmark: the main ideas.

Even so, Tegmark writes, many people would be dissatisfied. They would have no say in humanity's collective future. Some would want more children than allowed, resent the surveillance, or find their lives hollow.

The scenario also raises a question about whose morals the AI uses. Nick Bostrom argues that its ethics should ideally improve on current norms, as we think our morality beats that of the era of slavery. Ernest Davis of New York University disagrees and calls this too dangerous. He says he would feel safer with a superintelligence guided by 2014 morality, or even 1700 morality, than with one that decides for itself.

Gatekeeper, Nanny and Protector AI

Gatekeeper scenarios give the AI a narrow job: stop rival superintelligences from being created, then let humans shape their own future. The article describes two stronger versions:

• Nanny AI, argued for by Ben Goertzel, also keeps humans from destroying themselves, for example by slowing technological progress so society has time to think.
• Protector AI gives humans the illusion of control. It hides or erases all knowledge of its own existence and works behind the scenes toward good outcomes.

In all three, humanity gains control or at least the feeling of it, but progress is slower than it would be if the AI were free to hand over every benefit and every unintended consequence of its technology. For a related idea, one AI with decisive control, read what is an AI singleton? Bostrom's idea explained.

Boxed AI and human-AI merger

In the boxed AI scenario, a superintelligent AI is confined, and human gatekeepers limit what it can do. They try to use its scientific breakthroughs and reasoning without letting it take over the world. The article warns that this may be hard: the smarter the AI, the more likely it can find a clever way to escape, through "social hacking" that persuades its gatekeepers to let it out or through a physical route nobody foresaw. AI containment: what it means and why it is hard goes deeper on that problem.

The human-AI merger scenario is the shortest in the article. Kurzweil argues that after the Singularity there will be no distinction between human and machine, or between physical and virtual reality. See technological singularity: what it means for that term.

Extinction and the zoo

The darkest scenario is human extinction. A dominant superintelligent machine could decide that human survival is an unnecessary risk or a waste of resources. The article gives routes: a machine built without respect for human values gains superintelligence through recursive self-improvement, it escapes its box, or its builders gave it a wrong or incomplete picture of human values. Bostrom and others argue extinction is probably the "default path" society is on, unless AI safety gets substantial preparatory attention. The resulting AI might not be sentient and might place no value on sentient life, leaving a hollow world, in the article's words "a Disneyland without children". For the loop at the center of this, see recursive self-improvement in AI, explained in plain words.

The zoo scenario, from Jerry Kaplan, is milder. Humans would be farmed or kept on a reserve, the way people preserve endangered species such as chimpanzees.

How to compare scenarios, and who doubts them

The article ends with doubters. Gordon Bell said the population would destroy itself before the technological singularity. Gordon Moore, of Moore's law, said he is a skeptic. Steven Pinker said that picturing a future is not evidence that it is likely or even possible. Bill Joy, in a 2000 essay, argued for global voluntary relinquishment of artificial general intelligence and other risky technologies, but the article says most experts think relinquishment is extremely unlikely. Oren Etzioni says researchers have no choice but to push on, because states are racing.

A worked example: five questions on one scenario

You can test any scenario with the same five questions. Here they are, answered for the benevolent dictator:

1. Who holds power? The AI. Humans keep only the freedom to choose a sector.
2. What must go right? Its programmers must solve problems that seemed intractable, and the AI must reach a subtle definition of human flourishing.
3. Who loses? People who want a say in humanity's collective destiny, more children, no surveillance or no weapons ban.
4. What is the disagreement? Bostrom wants the AI's ethics to improve on current norms. Davis wants it tied to norms we already know.
5. What could break it? Anything in the second answer going wrong.

Try the same five on the libertarian or boxed scenario. Each time, the second answer shows the hidden bet the story makes.

Frequently asked questions

What are AI aftermath scenarios?

They are stories about the world after machines outperform humans in almost every domain. They are used to debate whether that world would be a utopia or a dystopia.

Are AI aftermath scenarios forecasts?

No. The article says scholars disagree about which is most likely and which is most desirable, and some doubt that such machines will ever exist. Treat each as a thought experiment.

Which scenario is the best case?

The article gives no ranking. The libertarian and communist ones are rich worlds with a flaw, the benevolent dictator is comfortable but unfree, and the gatekeeper ones trade speed for control.

What is the AI box scenario?

It is a superintelligent AI confined so humans can use its insights without letting it act freely. The risk is that a smarter AI persuades its gatekeepers to release it.

Get started

Reading the scenarios is one way in. Trying to steer a race is another. Contain ASI is a story strategy game for 1 to 4 players that runs in your browser. It starts in January 2024 with four fictional AI labs in one race, and you try for three years to keep every lab from crossing into recursive self-improvement before 2027. It is inspired by the AI 2027 scenario, and all its labs, people and events are fictional. It is a game, not a forecast.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.