Asilomar AI principles: what they say

The Asilomar AI principles are 23 guidelines for AI research written at a conference in California in January 2017. They range from everyday research habits, like funding safety work, to the longest-term questions, like how to treat AI that improves itself and what superintelligence should serve. This post explains where they came from, what each group says in plain words, and how the same idea of careful containment goes back to an earlier conference at the same place. You also get a short exercise to turn a principle into something you can check. The principles are quoted from the Future of Life Institute's Asilomar AI Principles page, and the history from Wikipedia.

The Asilomar AI principles in brief

The Future of Life Institute, a nonprofit that works against large-scale risks from technology, organized the Asilomar Conference on Beneficial AI. It was held on 5 to 8 January 2017 at the Asilomar Conference Grounds in California. More than 100 thought leaders and researchers in economics, law, ethics and philosophy met to work out principles of beneficial AI. The result was the 23 Asilomar AI Principles.

The principles were published as an open letter. Wikipedia counts signatures from 1797 AI and robotics researchers and 3923 others; the institute's page shows 5720 signatures in all. AI researchers who signed include Yoshua Bengio, Max Tegmark, Stuart Russell and Eliezer Yudkowsky. The institute's page calls the principles one of the earliest and most influential sets of AI governance principles.

The earlier Asilomar conference

Asilomar was already famous for a 1975 meeting, the Asilomar Conference on Recombinant DNA. About 140 professionals, mostly biologists, met there to draw up voluntary guidelines for a new technology that joins DNA from different organisms. Two of their principles sound familiar today: containment should be an essential part of how an experiment is designed, and the strength of containment should match the estimated risk as closely as possible. Scientists had halted those experiments over safety fears, and after the guidelines they carried on.

Research issues: principles 1 to 5

The first group is about how AI research should be done:

• Research goal. The goal should be "not undirected intelligence, but beneficial intelligence."
• Research funding. Investment in AI should come with funding for research on its beneficial use, including what values AI should be aligned with.
• Science-policy link. AI researchers and policy-makers should talk constructively.
• Research culture. Cooperation, trust and transparency among researchers and developers.
• Race avoidance. "Teams developing AI systems should actively cooperate to avoid corner-cutting on safety standards."

That last one is the heart of our post on the AI race: when rivals hurry, safety is the first thing cut.

Ethics and values: principles 6 to 18

The largest group covers how AI systems should behave and whom they should serve. In short:

• Safety and transparency. Systems should be safe and secure throughout their working life (6); if one causes harm, it should be possible to find out why (7); any role in court decisions should be explainable to a competent human authority (8).
• Responsibility and values. Designers and builders share responsibility for how their systems are used (9). Highly autonomous systems should have goals and behaviors that align with human values throughout their operation (10), and respect dignity, rights, freedoms and cultural diversity (11).
• Privacy and liberty. People should be able to access, manage and control the data they generate (12), and AI on personal data must not unreasonably limit their liberty (13).
• Sharing. AI should benefit and empower as many people as possible (14), and its economic gains should be shared broadly (15).
• Control and power. "Humans should choose how and whether to delegate decisions to AI systems" (16). Power from controlling advanced AI should respect, not subvert, the civic processes society depends on (17). And an arms race in lethal autonomous weapons should be avoided (18).

Longer-term issues: principles 19 to 23

The last five reach furthest, and they are the ones closest to the ideas behind Contain ASI:

• Capability caution (19). "There being no consensus, we should avoid strong assumptions regarding upper limits on future AI capabilities."
• Importance (20). Advanced AI "could represent a profound change in the history of life on Earth" and should be planned for with matching care and resources.
• Risks (21). Catastrophic or existential risks from AI must get planning and mitigation in proportion to their expected impact.
• Recursive self-improvement (22). AI systems designed to recursively self-improve or self-replicate in a way that could lead to rapidly increasing quality or quantity "must be subject to strict safety and control measures."
• Common good (23). "Superintelligence should only be developed in the service of widely shared ethical ideals, and for the benefit of all humanity rather than one state or organization."

Principle 22 names the exact threshold the game is built around. If the term is new to you, read our plain explainer of recursive self-improvement in AI.

What came after

The same institute later moved from principles to more direct asks. In March 2023 it published an open letter calling for a six-month pause on training the most powerful systems, covered in our post on calls to pause AI. In October 2025, according to Wikipedia's article on the Future of Life Institute, its Statement on Superintelligence called for a prohibition on developing superintelligence, not lifted before there is "broad scientific consensus that it will be done safely and controllably" and "strong public buy-in".

People read that path in different ways. Some see the 2017 principles as a shared starting point that later work built on. Others, including critics of the 2023 letter quoted in the same article, argue that focusing on future risks can draw attention away from harms happening now. This post takes no side.

A worked example: turn a principle into a check

Principles are easy to agree with and hard to test. This exercise takes about 15 minutes and shows the gap.

1. Pick one principle. Take number 7, failure transparency: "If an AI system causes harm, it should be possible to ascertain why."
2. Write the question it answers. After a harm, can someone find the cause?
3. Write what you would need to see. For example: a record of what the system was given and what it did, kept long enough to look back at.
4. Write who checks it. The builder, an outside reviewer, or a court? Principle 8 asks for "a competent human authority" in court cases; principle 7 names no one.
5. Repeat with principle 22. What would "strict safety and control measures" look like for a system that improves itself? Who decides they are strict enough?

You will find that most principles say what should be true but not who checks it or how. That is not a flaw unique to Asilomar; it is the step every set of principles leaves for later.

Exploring the idea in Contain ASI

Principle 5 asks rivals not to cut corners, and principle 22 asks for strict control over self-improving AI. Contain ASI lets you try both under pressure, as fiction. It is a story strategy game for 1 to 4 players that runs in your browser and needs WebGL 2. Chapter 1, Before ASI, starts in January 2024 with four fictional AI labs in one race. As a researcher, research manager, CEO or government, you try for three years to keep every lab from crossing into recursive self-improvement before 2027.

The game says it is inspired by the AI 2027 scenario, and that all its labs, people and events are fictional. It is a story to think with, not a forecast of what real labs will do.

Frequently asked questions

What are the Asilomar AI principles?

They are 23 guidelines for AI research written at the Asilomar Conference on Beneficial AI in January 2017, grouped into research issues, ethics and values, and longer-term issues.

Who wrote the Asilomar AI principles?

They came out of the conference organized by the Future of Life Institute, where more than 100 researchers and thought leaders in economics, law, ethics and philosophy met. They were then published as an open letter for others to sign.

Are the Asilomar AI principles binding?

No source opened for this post describes any way they are enforced. They were published as an open letter that people chose to sign.

Which principle is about recursive self-improvement?

Principle 22. It says AI systems designed to recursively self-improve or self-replicate in a way that could lead to rapid growth must be subject to strict safety and control measures.

Get started

Want to see how hard principle 22 is to keep when four labs are racing? Open Contain ASI in your browser and start a campaign as a researcher, research manager, CEO or government.