Nate Soares: who he is and what he argues about AI

Nate Soares is an American author and AI researcher known for his work on existential risk from AI. He is the president of the Machine Intelligence Research Institute (MIRI), and in 2025 he co-wrote the book If Anyone Builds It, Everyone Dies with Eliezer Yudkowsky. This post explains who he is, how his work moved from research to public warning, and what his book argues, with the reviews set out side by side. The claims come from Wikipedia's Nate Soares article.

Who Nate Soares is

Soares leads MIRI, a nonprofit research organization based in Berkeley, California. Our post on MIRI explains what it does.

He studied computer science and economics, and earned a Bachelor of Science from George Washington University in 2011. He then worked as a research associate at the National Institute of Standards and Technology, and as a contractor for the United States Department of Defense, building software tools for the National Defense University. After that he worked at two large technology companies.

In 2014, he left the technology industry to become a research fellow at MIRI.

The alignment problem and the Corrigibility paper

In 2014, Soares co-wrote a paper called "Corrigibility", published in the proceedings of a 2015 AAAI workshop. His co-authors were Benja Fallenstein, Eliezer Yudkowsky and Stuart Armstrong.

Soares and Yudkowsky say this paper introduced the term "AI alignment problem". The article defines it as the challenge of making increasingly capable AI systems behave as intended. Note the word "increasingly": the definition is about AI systems as they become more capable, not only the ones we have today.

The paper's subject, corrigibility, has its own post on Electrified: Corrigibility: why an AI that accepts correction is hard to build.

Setting a research agenda

At MIRI, Soares was the lead author of the institute's research agenda. In January 2015, that agenda was cited in a paper called "Research Priorities for Robust and Beneficial Artificial Intelligence". The paper accompanied an open letter that asked AI scientists to prioritize research "not only on making AI more capable, but also on maximizing the societal benefit of AI".

Among that paper's priorities was "research on the possibility of superintelligent machines or rapid, sustained self-improvement (intelligence explosion)". If those terms are new to you, our post on the intelligence explosion explains the idea and why people disagree about it.

His papers

The article lists several of his papers, which show the kind of questions he works on:

• "Corrigibility" (2015), with Fallenstein, Yudkowsky and Armstrong.
• "Agent Foundations for Aligning Machine Intelligence with Human Interests: A Technical Research Agenda" (2017), with Fallenstein.
• "A Formal Approach to the Problem of Logical Non-Omniscience" (2017), with four co-authors.
• "The Value Learning Problem" (2018), a chapter in the book Artificial Intelligence Safety and Security.
• "Cheating Death in Damascus" (2020), with Benjamin Levinstein, in The Journal of Philosophy.

From research to public warning

In 2015, Soares became MIRI's executive director, taking over from Luke Muehlhauser. In 2017, he gave a talk setting out open research problems in AI alignment and arguing that the alignment problem was especially difficult.

In 2023, MIRI changed course. It shifted its focus from alignment research toward warning policymakers and the public about the risks of possible future AI, while still supporting technical research. At the same time, Soares moved from executive director to president, and Malo Bourgon became MIRI's CEO.

In 2025, Soares told The Atlantic that he was focusing more on public awareness than on technical research. His reason, as the article reports it: AI development was moving too fast for adequate safeguards to be put in place.

If Anyone Builds It, Everyone Dies

In 2025, Soares and Yudkowsky published If Anyone Builds It, Everyone Dies. The article sums up its argument in three parts:

• Building AI far more capable than humans "using anything remotely like current techniques" would very likely lead to human extinction.
• AI alignment research is still at an early stage.
• International regulation will likely be needed to stop developers from racing to build systems the authors say could be catastrophically dangerous.

Our post If Anyone Builds It, Everyone Dies: the book explained goes through the book in more detail, and our post on Eliezer Yudkowsky covers his co-author.

How reviewers saw it

The reviews were mixed. The Guardian made it its "Book of the day" and called it "clear", though it said the conclusions are "hard to swallow". The Washington Post called it a polemic that offers few concrete instructions. The New Yorker featured it in its "Briefly Noted" section. This post takes no side.

A worked example: test the book's argument one step at a time

A strong claim is easier to think about when you split it into parts. Take the book's three parts and, for each one, write down what would change your mind.

1. "Current techniques would very likely lead to extinction." Ask: what would you need to see to believe a much more capable system built this way would behave as intended?
2. "Alignment research is at an early stage." Ask: what would count as real progress? A method that works on today's systems, or one that keeps working as they get more capable?
3. "Regulation will be needed to stop a race." Ask: if one developer slowed down alone, what would the others do?

You may agree with one step and not another. That is the point: it shows you exactly where you and the authors part ways. The third step is the one our post on the AI race looks at more closely.

Exploring the idea in Contain ASI

Contain ASI is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027.

It is a game, not a forecast: the page says all its labs, people and events are fictional, and it does not mention Soares or his book. But it lets you play out the third step of his argument, a race between developers, and see how hard it is to slow from the inside.

Frequently asked questions

Who is Nate Soares?

He is an American author and AI researcher, known for his work on existential risk from AI, and the president of the Machine Intelligence Research Institute (MIRI).

What did Nate Soares write with Eliezer Yudkowsky?

They co-wrote the 2025 book If Anyone Builds It, Everyone Dies, and both were co-authors of the "Corrigibility" paper, published in 2015.

Did Nate Soares coin the term AI alignment problem?

Soares and Yudkowsky say their "Corrigibility" paper introduced the term, meaning the challenge of making increasingly capable AI systems behave as intended.

What is Nate Soares's role at MIRI?

He joined as a research fellow in 2014, became executive director in 2015, and moved to president in 2023, when MIRI turned toward warning policymakers and the public.

Get started

Want to see how hard it is to slow a race from the inside? Open Contain ASI in your browser and start a campaign as a researcher, research manager, CEO or government.