Future of Humanity Institute: what it was and what it did

The Future of Humanity Institute, or FHI, was a research centre at the University of Oxford that studied big questions about humanity's long-term future. It treated the risk of human extinction, including risk from advanced AI, as a subject for academic research. It ran from 2005 until Oxford closed it in April 2024. This post explains who ran it, what it studied, and why its legacy is debated. The claims come from Wikipedia's Future of Humanity Institute article.

What the Future of Humanity Institute was

FHI was an interdisciplinary centre, which means it brought together people from different fields. It sat inside Oxford's Faculty of Philosophy and the Oxford Martin School. Its stated aim was to focus research where it could make the greatest positive difference for humanity in the long term.

Three names come up again and again:

• Nick Bostrom: the philosopher who founded it in November 2005 and served as its director.
• Anders Sandberg: a futurist on its research staff.
• Toby Ord: a senior research fellow, and the founder of the charity Giving What We Can.

FHI shared an office and worked closely with the Centre for Effective Altruism. Its work mixed academic research with outreach: it tried to encourage informed public discussion in government, business, universities and other organizations.

A short history, 2005 to 2024

Between 2008 and 2010, FHI hosted the Global Catastrophic Risks conference, wrote 22 academic journal articles and published 34 chapters in academic books. Its researchers gave policy advice at the World Economic Forum, to groups such as the MacArthur Foundation and the World Health Organization, and to government bodies in five countries, including the United Kingdom and the United States.

In 2009, Bostrom and the bioethicist Julian Savulescu published Human Enhancement. Over time, the institute turned more and more to the dangers of advanced artificial intelligence. In 2014, its researchers published several books on AI risk, including Stuart Armstrong's Smarter Than Us and Bostrom's Superintelligence: Paths, Dangers, Strategies. Our post on Superintelligence by Nick Bostrom covers that book's ideas.

In 2020, Ord published The Precipice, which argues that safeguarding humanity's future is among the most important moral issues of our time. See our post on The Precipice by Toby Ord.

On 16 April 2024, the University of Oxford closed the institute. FHI said it had "faced increasing administrative headwinds within the Faculty of Philosophy".

Existential risk: the institute's biggest subject

The largest topic FHI worked on was global catastrophic risk, and above all existential risk. In a 2002 paper, Bostrom defined an existential risk as one "where an adverse outcome would either annihilate Earth-originating intelligent life or permanently and drastically curtail its potential".

That second half matters. An existential risk does not have to kill anyone directly. Bostrom's 2003 paper "Astronomical Waste" discussed a future in which humanity survives but never spreads into space or uses the universe's resources for anything humans would value. By his definition, losing that potential counts too. Our post on existential risk from AI sets out the arguments for and against taking it seriously.

Natural risks and risks from technology

Bostrom and Milan Ćirković's 2008 book Global Catastrophic Risks collected essays on both kinds. Natural risks include supervolcanoes, asteroid impacts, gamma-ray bursts, cosmic rays, solar flares and supernovae. The article says these are seen as relatively small and well understood, with pandemics as a possible exception.

FHI paid more attention to risks people create: engineered pandemics, climate change, nuclear war and nuclear terrorism, molecular nanotechnology and artificial general intelligence. It expected the largest risks to come from future technologies, and from advanced AI in particular. On this, the article says, it agreed with the Centre for the Study of Existential Risk at Cambridge and the Machine Intelligence Research Institute. It also studied quieter risks that come with progress, such as totalitarianism, unemployment driven by automation, and information hazards.

Strange questions about our place in history

FHI spent much of its time on unusual threats that few other groups studied, and on the methods used to think about them. One theme was what our own position in history can tell us.

• The doomsday argument: it is unlikely that you happen to live at an extremely early point in human history. So, the argument says, you are probably near the middle of everyone who will ever live, which would mean humanity ends sooner than we might hope.
• The simulation argument: another idea FHI worked on, which Bostrom popularized.
• The Great Filter: a recurring theme was the Fermi paradox, the puzzling absence of visible alien civilizations. Robin Hanson argued there must be a "Great Filter" that stops civilizations from spreading through space. It might lie behind us, if intelligent life is very rare, or ahead of us, if existential risks are larger than we think.

The last idea ties straight back to AI. If the filter lies ahead, it would be the kind of existential risk FHI studied, and FHI expected the largest of those to come from advanced AI.

Human enhancement and better thinking

FHI also studied the promise and risks of changing humans themselves. Its bioethics work looked at gene therapy, life extension, brain implants, brain-computer interfaces and mind uploading. In 2008, Bostrom and Sandberg wrote a Brain Emulation Roadmap.

A related thread was human reasoning. FHI studied cognitive biases and ways to measure and improve human intelligence and rationality, as a way to shape the speed and direction of progress.

Why its legacy is debated

FHI's ideas spread widely, but not everyone welcomed them. Twelve days after the closure, The Observer ran a piece by Andrew Anthony titled "'Eugenics on steroids': the toxic and contested legacy of Oxford's Future of Humanity Institute". The article lists the piece as further reading and does not summarize it, so this post does not either, and takes no side.

A worked example: sort a risk the way FHI did

FHI's work gives you two simple questions for any big risk you read about. Try them on three examples.

1. Is it natural or made by people? A supervolcano is natural. An engineered pandemic and advanced AI are made by people.
2. Is it well understood? FHI's work described most natural risks as relatively small and well understood. Risks from new technology are less understood, which is why FHI expected the largest risks there.
3. Would it end humanity's potential, or only harm it? Use Bostrom's definition: a risk is existential if it could wipe out intelligent life from Earth, or permanently and drastically cut short what it could become. A terrible but recoverable disaster is catastrophic, not existential.

Now try it on advanced AI. It is made by people, it is poorly understood, and FHI's researchers argued it could be existential. That is why so much of the institute's later work was about AI. You can read more in our post on global catastrophic risk.

Exploring the idea in Contain ASI

Contain ASI is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027.

It is a game, not a forecast: the page says all its labs, people and events are fictional, and it does not mention FHI. But it puts you inside the kind of problem FHI studied: a technology made by people, poorly understood, moving fast.

Frequently asked questions

What was the Future of Humanity Institute?

It was a research centre at the University of Oxford, founded by Nick Bostrom in 2005, that studied big-picture questions about humanity's future, above all existential risk and the dangers of advanced AI.

Why did the Future of Humanity Institute close?

The University of Oxford closed it on 16 April 2024. The institute said it had faced increasing administrative headwinds within Oxford's Faculty of Philosophy.

Who worked at the Future of Humanity Institute?

Its director was Nick Bostrom. Its research staff included Anders Sandberg and Toby Ord, and Stuart Armstrong wrote one of its 2014 books on AI risk.

Is the Future of Humanity Institute the same as the Future of Life Institute?

No. FHI was a research centre at Oxford. The Future of Life Institute is a separate nonprofit that campaigns on risks from technology.

Get started

Want to feel how hard it is to steer a risky technology? Open Contain ASI in your browser and start a campaign as a researcher, research manager, CEO or government.