Existential risk studies: what the field is and who debates it

If you have seen the phrase existential risk studies and wondered what it covers, the short answer is this: it is a field of research that defines "existential risks", works out what they mean ethically, and looks at strategies for long-term survival. This post follows the field's story, from its roots in 19th-century thought on extinction to the arguments researchers now have about their own foundations. It sets out each side and gives no verdict.

What existential risk studies is

Existential risk studies (ERS for short) is a field focused on defining and theorizing existential risks, on their ethical implications, and on the related strategies of long-term survival.

The term "existential risk" has more than one definition. The wide idea is a global calamity that could cause the extinction of intelligent life on Earth, such as humans, or at least a severe limit on its potential. How that limit is defined is itself a topic inside the field.

The field also changes over time. Its source article splits its growth into waves, by how its ideas changed and how it related to neighbours such as futures studies, disaster studies, AI safety, effective altruism and longtermism.

Where the idea came from

The idea that humanity could simply end is newer than it feels. The source describes human extinction as a recent invention in the history of calamity, because it is thought of as an absolute and natural event, a break from older religious and mythological ideas of the end of the world.

Fiction led the way. Lord Byron's poem "Darkness" imagines a lifeless Earth. Mary Shelley's novel The Last Man shows one man living through the death of the rest of humanity. H. G. Wells wrote two essays on extinction, in 1893 and 1897. Isaac Asimov wrote a nonfiction book on the subject in 1979, A Choice of Catastrophes.

The other root is nuclear weapons. Early worries were atmospheric ignition, soon dismissed as implausible, and radioactive contamination. Scientists organized in response. The Russell-Einstein Manifesto of 1955 warned that human extinction was possible. The Pugwash Conferences were set up to reduce the threat of armed conflict. The Bulletin of the Atomic Scientists, formed by former members of the Manhattan Project, keeps the Doomsday Clock, which tracks global catastrophic risk as a point in time.

The three waves of the field

ERS took shape as its own field in the early 2000s. It then grew quickly in universities and with the public, helped by popular books, and it led to foundations, research centers and think tanks, some with substantial donor funding. Our posts on the Future of Humanity Institute and the Centre for the Study of Existential Risk cover two of them.

• First wave. The source dates the foundation to Nick Bostrom's 2002 essay "Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards". It framed human extinction as a question for analytic and utilitarian philosophy. It also tied the study of risk to a hope that large technological development could let humans survive long term, including by colonizing space. The source calls this technological utopianism, with its hope placed in technology, particularly artificial intelligence, genetic engineering and nanotechnology. The Centre for the Study of Existential Risk followed in 2012.
• Second wave. Researchers built on Bostrom's work and grew closer to effective altruism. The emphasis on transhumanism is described as reduced.
• Third wave. Once the field had consolidated, it turned to questions about who takes part and about the need for more than one theoretical approach. Some scholars began examining what they call the "historically dominant" approach.

Core terms: existential risk and its four outcomes

The canonical definition comes from Bostrom: a risk "where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential". That makes it both global and terminal.

The definition draws a line. Genocides and enslavement are called "local terminal risks". Moderate global warming, threats to biodiversity and economic recessions are "global endurable risks". Neither is existential in this strict sense. Some scholars found that scale too narrow and prefer the broader "global catastrophic risks", the title of a book Bostrom edited. Toby Ord is named as one who stays with the canonical definition. Our post on global catastrophic risk covers the wider term.

The 2002 essay also sorts outcomes into four categories named after T. S. Eliot's poem "The Hollow Men":

A "Bang" is sudden extinction. A "Crunch" is humanity's potential to become posthuman being frustrated, even if people live on. A "Shriek" is a limited form of posthumanity, and a "Whimper" is one that lacks meaningful values.

Bostrom also gave the field a rule of thumb, "Maxipok": maximize the probability of an okay outcome, where an okay outcome is any outcome that avoids existential disaster.

How effective altruism and AI fit in

The source says ERS has a substantial relation with effective altruism, the charity-giving philosophy and community. The field took on many of its ideas, drew effective altruists into its work, and the effective altruism community helped pay for its academic consolidation. Our post on effective altruism and AI risk goes into that link.

Artificial intelligence enters at two points in the source. One is as a first-wave technology on which the field placed its hopes. The other is as a named neighbour: AI safety is one of the areas the field relates to, and the article lists existential risk from artificial general intelligence among its related topics. For the AI arguments themselves, see our post on existential risk from AI.

The main critiques

The source sets out three lines of criticism, some of them from scholars inside the field.

• The techno-utopian core. These critics say the original base rests on three assumptions, not all shared by every adherent. First, that a maximally developed future could hold enormous value through more fulfilling posthuman ways of living. Second, that failing to reach it would be an existential catastrophe. Third, that today's moral duty is to bring that future about, "including through exceptional actions". They link this to transhumanism, longtermism and total utilitarianism, and say it shows in Bostrom's essays and in books such as The Precipice, Superintelligence and What We Owe the Future. They call for new frameworks and more democratic values.
• History and politics. Others say the concept is too narrow and overlooks the history of genocides, especially the colonial genocide of indigenous peoples. The source notes Bostrom starts human-made existential risk after World War II, with nuclear weapons. Hobson and Corry, in a 2023 article, ask whether existential security would lead to shared emergency measures or erode normal politics and legitimize the interests of a few posing as those of humanity.
• The neglect claim. Bostrom wrote in 2013 that one database held 900 papers on dung beetles but fewer than 50 on "human extinction". Schuster and Woods repeated the search in March 2020 and found slightly better numbers for human extinction, but 7,166 papers on "genocide". On JSTOR they report 66,809 results for "human extinction", 43,926 for "genocide" and 134,089 for "extinction". They say the gap reflects Bostrom's self-defined terms.

The source gives the critics' claims, not replies to them, so read them as claims by their authors. Our posts on longtermism and AI and on Nick Bostrom give more background.

A worked example: read an existential risk claim in four questions

Say you meet the line "this threat is an existential risk". These four questions come straight from the field's own terms.

1. Which definition is in use? Does the claim mean the strict, canonical one (annihilation or a permanent, drastic loss of potential) or the wider "global catastrophic" one? A serious harm that is endurable is not existential in the strict sense.
2. Which outcome is meant? Is it a Bang (sudden extinction), or one of the quieter ones: a Crunch, a Shriek or a Whimper, where people may live on but the future is cut short?
3. What is it assuming? Does the argument lean on the three techno-utopian assumptions, such as a huge future value that we have a duty to secure? If so, a reader who rejects them may reject the conclusion without disputing the facts.
4. Whose history is left out? Does the claim treat past harms, such as genocides, as outside its scope? The critics say that choice is a political one, and worth noticing.

Run any headline through these four and you can say what is claimed and what you would need to accept to agree.

Frequently asked questions

What is existential risk studies?

It is a field focused on defining and theorizing existential risks, on their ethical implications and on strategies for long-term survival. It emerged as a distinct field in the early 2000s.

Who founded the field?

Its foundation is dated to Nick Bostrom's 2002 essay on analyzing human extinction scenarios. The source also traces earlier roots to 19th-century thought on extinction and Cold War work on global catastrophic risk.

What is the difference between existential risk and global catastrophic risk?

The canonical definition covers only outcomes that annihilate intelligent life or permanently and drastically curtail its potential. "Global catastrophic risk" is a broader category that some scholars prefer.

Is existential risk studies only about AI?

No. Its roots lie in extinction fiction and nuclear weapons, and AI is one technology and one neighbouring field among several the source names.

What do critics say about the field?

They raise three points: its techno-utopian assumptions, its narrow view of history and politics, and the claim that its subject is neglected, which they say rests on self-defined search terms.

Get started

Reading about a field is one thing; thinking through a race is another. Contain ASI is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. It starts in January 2024 with four fictional AI labs in one race, and for three years you try to keep every lab from crossing into recursive self-improvement before 2027.

The page says it is inspired by the AI 2027 scenario and that all its labs, people and events are fictional. It is a game, not a forecast and not a course.