Calls to pause AI: what they ask and who disagrees

The calls to pause AI development ask one thing in different ways: slow down or stop training the most powerful AI systems until people know how to make them safe. The best known are a 2023 open letter signed by more than 30,000 people and a protest movement called PauseAI. This post explains what each one asks for and why, what happened next, and the main objections, with no verdict. It ends with an exercise to weigh any pause proposal yourself. Every claim about the calls comes from Wikipedia's articles on the open letter and PauseAI.

What the calls to pause AI ask for

The calls share three ideas:

• A pause on the biggest training runs, not on all AI: the targets are the most powerful general systems.
• A condition for restarting: knowing how to build such systems safely.
• Coordination, because one group pausing alone does little if others keep going.

They differ on how long, who decides, and how to enforce it. The two main calls show the range.

The 2023 open letter: a six-month pause

In March 2023, the Future of Life Institute published Pause Giant AI Experiments: An Open Letter. It came a week after a leading AI lab released a new large language model. The letter called on all AI labs to immediately pause, for at least six months, the training of AI systems more powerful than that model.

Its reasons included AI-generated propaganda, extreme automation of jobs, human obsolescence and a society-wide loss of control. It said current models were "becoming human-competitive at general tasks". It worried about race-to-the-bottom dynamics, where labs might skip safety to ship products faster. That is the same pressure described in AI race: what it is and why it worries researchers.

The letter asked researchers to refocus on making powerful AI "more accurate, safe, interpretable, transparent, robust, aligned, trustworthy, and loyal". It also recommended:

• more government regulation;
• independent audits before training AI systems;
• "tracking highly capable AI systems and large pools of computational capability";
• "robust public funding for technical AI safety research".

It suggested using the amount of computation that goes into a training run as a rough measure of how powerful a system is, and so as the line for what counts.

Who signed it

More than 30,000 people signed. Among them were the Canadian AI researcher and Turing Award recipient Yoshua Bengio, the computer scientist Stuart Russell, the historian and philosopher Yuval Noah Harari, Gary Marcus of New York University, and the philosopher Huw Price, a co-founder of the Centre for the Study of Existential Risk. Max Tegmark, whose institute published it, also signed; his own Wikipedia article says he advocates for a halt on the development of artificial superintelligence (Max Tegmark on Wikipedia).

PauseAI: a movement for a global pause

PauseAI is a political movement founded in Utrecht, in the Netherlands, in May 2023 by the software entrepreneur Joep Meindertsma. Its stated aim is global coordination to stop the development of more powerful general AI, at least until it is known how to build such systems safely, and to keep them under democratic control. Its stated goal is to "implement a temporary pause on the training of the most powerful general AI systems".

Its proposed steps:

1. Set up an international AI safety agency, similar to the International Atomic Energy Agency.
2. Allow training of general AI systems only when their safety can be shown.
3. Allow deployment of models only when no dangerous capabilities are present.

Meindertsma has said he first grew worried after reading Nick Bostrom's 2014 book Superintelligence (see Superintelligence by Nick Bostrom: the main ideas). PauseAI protested outside the first AI Safety Summit at Bletchley Park in November 2023. Meindertsma called the summit's declaration a small first step and argued that binding international treaties are needed, pointing to the Montreal Protocol and the ban on blinding laser weapons as past successes. On 13 May 2024, before the AI Seoul Summit, the movement held protests in thirteen countries, with signs reading "When in doubt, pause".

What happened next

According to the letter's article, a pause had not happened as of July 2024. On the letter's first anniversary, the Future of Life Institute itself said that AI companies had instead directed "vast investments in infrastructure to train ever-more giant AI systems".

The letter was still credited with creating "renewed urgency within governments to work out what to do about the rapid progress of AI", and with reflecting growing public concern about AI risks.

The main objections to a pause

The calls drew criticism from several directions, which do not agree with each other:

• It does not go far enough. Eliezer Yudkowsky wrote that the letter "doesn't go far enough" and should ask for an indefinite pause, since solving the alignment problem might take decades.
• It distracts from harms today. Critics said it pulled attention from present problems such as biased algorithms. Timnit Gebru and others called it sensationalist, saying it amplified "some futuristic, dystopian sci-fi scenario".
• It cannot be enforced. One AI ethicist said the letter raised awareness of real issues, such as voice cloning, but was unactionable and unenforceable.
• It asks the wrong people. One prominent critic said that asking one particular group to pause does not solve the challenges; another called the letter "virtue signalling" with no real effect.

On the other side, some IEEE members who signed said "There are too many ways these systems could be abused," with no review or regulation in place to prevent harm. For the wider debate on how serious the risk is, see Existential risk from AI: the main arguments for and against, and for an approach that steers rather than stops, differential technological development.

A worked example: weigh a pause proposal in five questions

When you read a new call to pause AI, these five questions, built from the proposals and objections above, help you judge it on its own terms:

1. What exactly stops? The letter used training computation as the line. Is the line clear enough that people could check it?
2. For how long, and what restarts it? Six months, or until safety is shown? Who decides that safety is shown?
3. Who has to join? If only some groups pause, does the race simply move? This is the coordination problem behind the call for an international agency.
4. How is it enforced? Audits, tracking of large pools of computing power, or a treaty like the Montreal Protocol? The "unenforceable" objection lives here.
5. What does it cost today? Does attention to future risks crowd out work on present harms, as critics say, or can both be done?

Try it on the open letter first. Question 1 has a clear answer in the letter, training computation; questions 3 and 4 are where the "asks the wrong people" and "unenforceable" objections apply.

Frequently asked questions

Did the AI pause happen?

No. As of July 2024 the pause the letter asked for had not been realized, though the letter was credited with pushing governments to act with more urgency.

What is PauseAI?

A political movement founded in the Netherlands in May 2023 that wants a temporary pause on training the most powerful general AI systems until they can be built safely.

Why do some people oppose a pause?

Some say it goes too far or cannot be enforced, some say it distracts from harms happening now, and some say it does not go far enough.

Is Contain ASI about the pause letter?

No. The game's page names the AI 2027 scenario as its inspiration and says all its labs, people and events are fictional.

Get started

The calls to pause AI are about coordination under race pressure. Contain ASI lets you try that pressure as fiction. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027. It is a game, not a forecast, and it runs in your browser on WebGL 2, so use a recent Chrome, Edge, Firefox or Safari.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.