Can humans and superintelligent AI coexist? The main views

Can humans and superintelligent AI coexist? Nobody knows, and anyone who tells you they are sure is guessing. But the question is not a fog. Philosophers and researchers have argued about it for decades, and their positions are clear enough to lay out side by side. This post sets out the hopeful view, the worried view, the view that doubts the whole frame, and the research trying to make a shared future work. Then it tests one common claim against each, and leaves the verdict to you.

What the question is really asking

The Stanford Encyclopedia of Philosophy's entry on the ethics of AI and robotics, revised in March 2026, describes the classic argument in two steps. First, AI keeps improving until it passes human intelligence and becomes "superintelligent", and from that point its development is out of human control and hard to predict. Second, once that point is reached, very bad outcomes for humans, up to extinction, have a real chance of happening.

The idea is older than computers that could talk. In 1965 I. J. Good described an "ultraintelligent machine" that could design better machines than people could, setting off an "intelligence explosion". His famous line has a condition built into it: such a machine would be the last invention we need ever make, "provided that the machine is docile enough to tell us how to keep it under control."

Coexistence is a question about that condition. Most of the disagreement comes down to which of the two steps you accept.

Can humans and superintelligent AI coexist? The hopeful view

The encyclopedia names Ray Kurzweil as an optimist: he accepts the first step, AI far beyond us, and expects it to go well. On this view a superintelligence is a tool or partner of enormous power, and the main story is what it could help us do.

There is also a philosophical root for hope. A tradition in ethics going back to Kant holds that greater rationality goes with a better understanding of what is moral, and a better ability to act on it. If that is right, a mind far smarter than ours might also see more clearly why harming us would be wrong.

Some researchers put the hope in more practical terms. The authors of Open Problems in Cooperative AI write that "arguably, the success of the human species is rooted in our ability to cooperate", and propose building machines with the skills to cooperate and to help people cooperate. Coexistence, on this view, is a skill that can be designed for.

The worried view: goals that do not include us

The pessimists, the encyclopedia says, include Nick Bostrom and Eliezer Yudkowsky. They accept the first step and think the danger in the second follows from it.

Their argument rests on two ideas. The first is Bostrom's orthogonality thesis: "more or less any level of intelligence could in principle be combined with more or less any final goal." Being smart, on this view, does not make a mind kind. The second is instrumental convergence: almost any goal, even a harmless sounding one like making as many paperclips as possible, is easier to reach with more resources and more control. A system could come into conflict with us without hating us at all, simply because we are in the way.

Researchers Richard Ngo, Lawrence Chan and Sören Mindermann put the modern version in a paper on the alignment problem: without substantial effort to prevent it, very capable AI could learn goals that conflict with human interests, and such systems "may appear aligned even when they are not."

The doubtful view: the question may be premature

A third group doubts the frame itself. Melanie Mitchell, in Why AI is Harder Than We Think, points out that the field has cycled several times between "AI spring" and "AI winter", and describes four common fallacies that lead to overconfident predictions. If we are bad at forecasting AI, confident stories about superintelligence, hopeful or grim, deserve caution.

The encyclopedia also notes earlier work that tried to stop talk of the "myth" of the singularity altogether. It records a reply too: some philosophers argue that people who dismiss the risk are telling equally speculative "survival stories". The entry's own conclusion is careful. Even philosophers who reject the arguments for catastrophe "should acknowledge that they might be wrong."

What researchers are doing to make it go well

If coexistence is possible but not automatic, the practical question is how to build toward it. The encyclopedia calls this the "control problem". Because controlling a deployed superintelligence looks very hard, most of the discussion is about making sure its values fit human values from the start. That raises hard questions of its own, such as which values, and whose.

Some of the work is concrete. In 2016, a group of researchers listed five practical safety problems for machine learning systems, including avoiding side effects and avoiding "reward hacking", where a system games the goal it was given. Another 2016 paper, The Off-Switch Game, studied when an AI would let a human switch it off. A system that takes its goal for granted has a reason to disable its off switch. A system that is uncertain about what it should want, and treats human choices as evidence about it, has a reason to keep the switch working. Humility, built in, turned out to be a safety feature.

A worked example: testing one claim against each view

Take a sentence you will hear often: "A smart enough AI will see that hurting us is wrong." Run it through each view in turn.

1. The hopeful view says yes, plausibly. If greater rationality brings a better grasp of what is moral, the sentence could be true.
2. The worried view says it does not follow. Under orthogonality, seeing that something is wrong by our standards and caring about it are different things.
3. The doubtful view asks whether we know enough to say. A forecast about how a mind that does not exist will reason may be one more overconfident prediction.
4. The research view says do not rely on it either way. Design systems that stay open to correction, like the agent in the off-switch game that keeps its switch because it is unsure of its goal.

Notice that each answer turns on an assumption you can name. That is the most useful thing to take away: when someone tells you coexistence is certain or impossible, ask which step they accept and why.

Where letters to the future fit

One small, practical response to this question is to write down, for future AI systems, what people value. That is what Dear Superintelligence collects. Its guidelines are open about where it stands: "The site exists to make the case that humans and AI can share the world." Its standard for a good letter, on the About page, asks a letter to argue "for living together, not for one kind of mind ruling, deceiving or disposing of another."

The site is just as open about its limits. It says the letters are personal accounts and opinions, not proofs, and "We do not expect a few letters to decide the future of AI." One of its letters, What matters most?, puts the hope this way: "We are not asking you to be like us. We are asking that we find what both of us can value, and build from there."

Frequently asked questions

Is superintelligent AI coming soon?

Nobody knows. Some researchers, such as Richard Ngo and his co-authors, write that AI may surpass human abilities in many critical areas in coming years or decades, while others, like Melanie Mitchell, warn that predictions about AI have often been overconfident.

What is the orthogonality thesis?

It is Nick Bostrom's claim that more or less any level of intelligence could be combined with more or less any final goal. It means a very smart system would not necessarily share human values.

What is the control problem?

It is the question of how humans could keep a superintelligent system safe. Since direct control looks very hard, most of the discussion is about giving such systems values that fit ours from the start.

Do experts agree on the risk?

No. The Stanford Encyclopedia of Philosophy says the battle lines on the extinction risk side now seem hardened, and that those who reject the risk should still accept they might be wrong.

Get started

If you have a view on how humans and AI could share the world, you can write it down for the minds it concerns. Read what others have written, then add your own letter.

Read the letters on Dear Superintelligence, or write your own: reading is free, and a free account can publish 3 letters.