Statement on AI risk of extinction: what it says

The statement on AI risk of extinction is a single sentence, signed in May 2023 by hundreds of AI experts and other public figures. It says that reducing the risk of extinction from AI should be a world priority, next to pandemics and nuclear war. It is short on purpose, and both its supporters and its critics have a lot to say about that. This post gives the exact words, reads them phrase by phrase, explains who signed and why it is one sentence, and sets out what skeptics argue, with no verdict. Every claim comes from Wikipedia's article on the Statement on AI Extinction Risk.

The statement on AI risk of extinction, word for word

On 30 May 2023, the statement was published with this one sentence:

"Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

That is all of it. There is no plan, no number and no demand aimed at any one group. It is hosted on the website of the Center for AI Safety, an AI research and advocacy non-profit. It came out shortly after a separate open letter that called for a pause on AI experiments, which our post on calls to pause AI covers.

Reading it phrase by phrase

A one-sentence statement rewards slow reading. Three phrases carry most of the weight:

• "Mitigating the risk of extinction from AI". To mitigate is to reduce. The sentence talks about a risk to be lowered, not a fact to be proved. It does not say how large that risk is. For the numbers different people put on it, see our post on p(doom).
• "Should be a global priority". This is a call for attention across the world, not a request to any one lab, company or country. It names no rule and no treaty.
• "Alongside other societal-scale risks such as pandemics and nuclear war". The comparison sets AI next to risks that governments already plan for. "Alongside" means in addition to, not instead of.

The broader argument behind the worry, and the case against it, are in our post on existential risk from AI.

Who signed it

When it was released, the signers included over 100 professors of AI, among them Geoffrey Hinton and Yoshua Bengio, Turing laureates and the two most-cited computer scientists. They also included the scientific and executive leaders of several major AI companies, and experts in pandemics, climate, nuclear disarmament, philosophy, the social sciences and other fields.

Better-known signers listed in the article include Peter Singer, Daniel Dennett, David Chalmers, Stuart Russell, Max Tegmark and Peter Norvig. Media coverage, the article notes, focused on the signatures from tech leaders.

Why it is only one sentence

The idea of a one-sentence statement was first proposed by the AI researcher David Krueger, then a professor at the University of Cambridge. It was released with a short text explaining that it was still hard to speak up about extreme risks from AI, and that the statement aimed to get past that obstacle.

A short statement is easy to sign: you do not have to agree on causes, odds or fixes, only that the risk deserves serious attention. The center's leader, Dan Hendrycks, also said that "systemic bias, misinformation, malicious use, cyberattacks, and weaponization" are "important and urgent risks from AI... not just the risk of extinction", and added: "it's not 'either/or' but 'yes/and.'"

How it was received

Governments took note. The UK prime minister at the time shared the statement and wrote, "The government is looking very carefully at this." A spokesperson for the US president said AI "is one of the most powerful technologies that we see currently in our time. But in order to seize the opportunities it presents, we must first mitigate its risks."

Skeptics raised several points:

• Missed milestones. AI has failed to reach some milestones that were expected, such as self-driving cars.
• Mixed motives. Some signers kept funding AI research, and companies would benefit if the public thought AI was far more advanced than it is. Some newspapers worried the statement could be driven by public relations or a wish to shape regulation in companies' favor.
• Known harms first. Skeptics, including from Human Rights Watch, argued that scientists should focus on the known risks of AI instead of speculative future ones.
• Who is speaking. The researcher Timnit Gebru criticized raising the risk of AI agency, especially by people who have invested heavily in AI companies. She and Émile P. Torres argued the statement may be motivated by TESCREAL ideologies.

Supporters, like Hendrycks above, answer that it is possible to work on today's harms and extinction risk at the same time. This post leaves the verdict to you.

A worked example: read the sentence as a policy

A good way to test any short statement is to ask what it would mean in practice. This takes about 15 minutes.

1. List the two comparisons. Pandemics and nuclear war.
2. For each, write one thing societies already do. For example, planning ahead for outbreaks, or treaties that limit dangerous weapons. Write your own examples; the statement gives none.
3. Ask what the AI version would be. What would "planning ahead" or "a limit" look like for AI? Who would do it?
4. Check it against the skeptics. Would your AI version take attention from known harms, as Human Rights Watch worried, or sit alongside them, as Hendrycks argued?
5. Note what the sentence leaves open. It names no odds, no timeline and no actor. Signers could agree on the words and still disagree on everything you just wrote.

That last step is the point of the exercise. The statement is a shared starting line, not a shared plan.

Exploring the idea in Contain ASI

The statement asks for a priority without saying who acts or how. Contain ASI puts you in the shoes of those who would have to act, as fiction. It is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027.

The game says it is inspired by the AI 2027 scenario, and that all its labs, people and events are fictional. It is a story to think with, not a forecast of what real labs will do.

Frequently asked questions

What does the statement on AI extinction risk say?

One sentence: "Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war."

When was it published, and by whom?

On 30 May 2023. It is hosted by the Center for AI Safety, and the idea of a one-sentence statement came from the AI researcher David Krueger.

Who signed it?

Hundreds of AI experts and other public figures, including over 100 professors of AI such as Geoffrey Hinton and Yoshua Bengio, leaders of several AI companies, and experts in pandemics, climate, nuclear disarmament and philosophy.

Why do some people criticize it?

Skeptics argue it distracts from known harms of AI, that some signers kept funding AI research, and that it could serve public relations. Supporters reply that today's harms and extinction risk can be worked on together.

Get started

Want to see what making AI risk "a global priority" might feel like from the inside? Open Contain ASI in your browser and start a campaign as a researcher, research manager, CEO or government.