The Precipice by Toby Ord: the book's main ideas

The Precipice by Toby Ord is a 2020 book arguing that humanity is living through a uniquely dangerous period, with risks it has never faced before. Its full title is The Precipice: Existential Risk and the Future of Humanity. This guide covers the book's main ideas, Ord's much-quoted risk numbers, a simple exercise to test your own view, and what reviewers made of it.

What is The Precipice by Toby Ord about?

Toby Ord is an Australian philosopher. In 2009 he founded Giving What We Can, a society whose members pledge at least 10 per cent of their income to effective charities, and he is a key figure in the effective altruism movement. He wrote The Precipice as a senior research fellow at Oxford's Future of Humanity Institute, which closed in 2024. The book came out in March 2020.

Wikipedia's article on The Precipice sums up its argument: humanity faces unprecedented risks over the next few centuries, and safeguarding its future matters enormously.

The title names a period of history. In Ord's telling, the Precipice began with the first atomic bomb test in 1945. Since then, humanity has had destructive power it never had before, without the wisdom and restraint to match. Ord expects the period to last no more than a few centuries: either humanity learns self-control fairly quickly, or the risk keeps building until something goes badly wrong.

He gives one example of how close it has come. In his estimate, the Cuban Missile Crisis of 1962, which leaders at the time thought had a 10 to 50 per cent chance of causing nuclear war, was the nearest humanity has come to destroying itself in its 200,000-year history.

What Ord means by existential catastrophe

Ord borrows two terms from Nick Bostrom, whose ideas are covered in the post on Superintelligence.

• An existential catastrophe is the destruction of humanity's long-term potential.
• An existential risk is the probability that a given hazard leads to one.

The definition is strict. Human extinction is one way it could happen, but not the only one: a permanent totalitarian dystopia would also count. Yet a disaster that killed most people would not count, as long as the survivors eventually recovered and carried on. What matters is whether the loss can ever be undone. That is close to the worry in value lock-in.

Ord argues such a catastrophe would be terrible in three ways at once: it would betray everything past generations built, harm the people alive at the time, and cut off a vast future that could flourish among the stars.

Ord's risk numbers, and why AI tops the list

The book is best known for its estimates. Ord puts the total risk of an existential catastrophe in the next century at 1 in 6. Natural risks, like asteroid impacts, are a small part of that. Almost all of it comes from things humans do: nuclear war, engineered pandemics, climate change and, above all, AI.

For unaligned artificial general intelligence, Ord's estimate is 1 in 10 over the next century, higher than all his other sources of existential risk combined. The post on p(doom) sets that figure beside other people's.

Ord also argues that the picture of an AI takeover with robots is a misconception driven by the media and Hollywood. The most damaging people in history, he notes, were not the strongest; they used words to win people over. A very capable AI with access to the internet could, in his view, copy itself to many places, gather money and helpers, persuade people on a large scale and exploit weaknesses too subtle for humans to see.

His conclusion is not to abandon AI. He treats the risk as a reason for "proceeding with due caution".

What Ord says humanity should do

Ord states that humanity spends less than 0.001 per cent of gross world product on work aimed directly at reducing existential risk. He argues that poor global coordination keeps that number low, and that specialized global institutions could help.

He backs the strategy of speeding up protective technologies. In his words: "While it may be too difficult to prevent the development of a risky technology, we may be able to reduce existential risk by speeding up the development of protective technologies relative to dangerous ones."

He also suggests things individuals can do: choose a career with high impact, give effectively, and add to the public conversation about these risks.

A worked example: build your own risk table

Ord's numbers are his own judgements, not measurements. A good way to understand them is to make your own and compare. It takes about 30 minutes.

1. List the risks. Write five rows: natural risks (like asteroids), nuclear war, engineered pandemics, climate change, and unaligned AI.
2. Give each a number. For the next 100 years, write your own chance that each one destroys humanity's long-term potential. Use Ord's strict definition: recovery means it does not count.
3. Write one reason per row. One sentence on why you chose that number.
4. Compare with Ord. His total is 1 in 6, about the chance of rolling a six on one die, and his AI number is 1 in 10. Where are you far from him, and why?
5. Pick one lever. For your biggest risk, write one protective step that could be sped up, following his idea above.

The exercise is not about getting the right answer. It shows how much these numbers depend on judgement, which is exactly why people disagree about them.

How the book was received

Many reviewers praised it. The Evening Standard called it a "startling and rigorous contribution". In The Spectator, Tom Chivers called it "a powerful book, written with a philosopher's eye for counterarguments so that he can meet them in advance". Steven Carroll in The Sydney Morning Herald found it authoritative and accessible. Published during the coronavirus pandemic, it struck The New Yorker as "made for the present moment".

There were doubts too. Writing in The Sunday Times, Bryan Appleyard was skeptical of some of its moral philosophy: "I doubt that it can redirect humanity away from its self-destructive ways." He still praised it as "dense and often thrillingly written" and called its analysis of the science "exemplary". For the wider debate over whether the far future should come first, see longtermism and AI.

Explore the stakes as fiction in Contain ASI

Ord's biggest single risk is the one Contain ASI asks you to hold back. Its home page reads: "January 2024. Four AI labs. One race. For three years, keep every lab from crossing into recursive self-improvement before 2027." It is a story strategy game for 1 to 4 players that runs in your browser, where you play as a researcher, research manager, CEO or government.

It is fiction, not a forecast. The footer says it is inspired by the AI 2027 scenario and that all its labs, people and events are made up.

Frequently asked questions

What is the main argument of The Precipice?

That since 1945 humanity has had the power to destroy itself without the wisdom to match, that this dangerous period could last a few centuries, and that reducing existential risk deserves far more attention than it gets.

What risk does Toby Ord give for AI?

He estimates a 1 in 10 risk of existential catastrophe from unaligned artificial general intelligence over the next century, out of a 1 in 6 total from all sources.

Does Ord say we should stop building AI?

No. He treats the risk as a reason for "proceeding with due caution", not for abandoning AI.

Is Contain ASI a forecast of the future?

No. It is a game, and its footer says all its labs, people and events are fictional.

Get started

Want to see how hard it is to keep four labs on the safe side of the line? Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional. You will need a recent Chrome, Edge, Firefox or Safari, because it runs on WebGL 2.