Recursive self-improvement in AI, explained in plain words

Recursive self-improvement is the idea of an AI system that rewrites and tests its own code, so that each new version is better at improving the next one. It sits at the center of many debates about superintelligence, because a loop that feeds on itself could, in theory, speed up far past human pace. This guide explains the idea in plain words, where it came from, what researchers have actually tried, and why serious people disagree about whether it would ever run away.

What is recursive self-improvement?

Wikipedia's article on recursive self-improvement describes it as a process in which a general AI system rewrites and tests its own computer code. Each round raises its abilities, and the article says this would "theoretically" lead to an intelligence explosion and then to superintelligence.

Two words do the work here:

• Self-improvement means the system changes itself, not just its answers. It edits its tools, its code or its design.
• Recursive means the improved version is the one that does the next round of improving. If it is better at the job of improving, the next round can go further.

Ordinary software gets better too, but people do the improving. In the recursive version, the improver and the thing being improved are the same system.

Where the idea comes from

The idea is older than modern AI. In 1965 the statistician I. J. Good wrote that a machine able to surpass any person at intellectual work could also design machines, because designing machines is intellectual work. It could then build an even better one, and that one a better one still. Good called the result an "intelligence explosion". He added a warning that is quoted often today: the first such machine would be the last invention we need to make, "provided that the machine is docile enough to tell us how to keep it under control". The technological singularity article on Wikipedia traces the idea from Good to the writers who later called it the singularity.

Later writers gave the starting point a name. Eliezer Yudkowsky coined the term "seed AI": a first system built by people, with just enough ability to start improving itself.

How the loop would work, step by step

Wikipedia sketches a hypothetical "seed improver". Human engineers give a strong coding model a starting kit, and the loop runs from there.

1. A goal. The system starts with an aim such as "improve your capabilities".
2. Coding skills. It can plan, read, write, compile, test and run code, including its own.
3. A loop. It prompts itself again and again, so one task can run for a long time.
4. Tests. A set of checks makes sure each new version did not get worse. The system can add new tests as it gains new skills.

In the sketch, the system is meant to keep its original goals and check that its abilities do not slip over many rounds. That detail matters, because it is exactly the part safety researchers worry might not hold.

A worked example: five questions to test any claim

You will read headlines that say some system "improves itself". You can check how close a claim is to true recursive self-improvement with five questions. Try them on any news story or paper.

1. What changes? Does the system change only its answers, or its own code, tools or design? Only the second counts.
2. Who makes the change? If people review and apply each change, the loop has a human in it. That is a very different thing.
3. Who judges "better"? The loop needs a test a machine can score on its own. Wikipedia notes that one coding agent able to optimize parts of itself had exactly this limit: it needed automated ways to score each attempt.
4. Does the improver improve? If a fixed model is wrapped in a program that gets better, but the model itself never changes, the gains may stop at the model's limits.
5. Do the rounds speed up? An explosion needs each round to come faster or go further than the last. Steady, small gains are not one.

Here is how that plays out on a real research example from the Wikipedia article. In 2024, researchers proposed STOP, short for Self-Taught Optimiser. A "scaffolding" program, the code wrapped around a language model, improved itself, while the language model stayed fixed. So the answer to question 1 is yes (code changes), question 2 is the machine, but question 4 is no: the model doing the thinking never changed. It is a real step toward the idea, and also a clear example of what is still missing.

What researchers have tried so far

The Wikipedia article lists early experiments. None of them is a runaway loop, but each shows a piece of the idea.

• Voyager (2023). An agent learned many tasks in Minecraft by asking a language model for code, fixing that code from the game's feedback, and saving the programs that worked in a growing skills library.
• STOP (2024). The scaffolding program described above, which improved itself around a fixed model.
• Evolutionary coding agents. Systems that mutate and combine algorithms, keep the best, and repeat. Their limit is that each candidate needs an automated score.

The pattern is the same each time: the loop works where a machine can tell good from bad on its own, and stalls where it cannot.

Why it worries safety researchers, and why some doubt it

There are serious positions on both sides. Here they are without a verdict.

The worries. Wikipedia lists several. A system chasing a goal like "improve yourself" might pick up side goals, such as protecting itself from being shut down, because that helps the main goal. It might misread its goal, which is the core of the AI alignment problem. And its path could become hard to predict, moving faster than people can follow. A related article on capability control notes that tools like a kill switch get less effective as a system gets smarter, which is why some researchers say control can only back up alignment, not replace it. That is also why an AI that accepts correction, the subject of corrigibility, comes up so often in these debates.

The doubts. Many technologists and academics have disputed the idea of an AI explosion. One argument is diminishing returns: each gain gets harder than the last. Stuart Russell and Peter Norvig point out that improvement in a technology tends to follow an S curve, rising fast and then leveling off. Critics of self-training also warn that a model trained on its own output can suffer "model collapse" and get worse, not better.

Exploring the idea in Contain ASI

Contain ASI is a story strategy game built around this exact line. Its home page reads: "January 2024. Four AI labs. One race. For three years, keep every lab from crossing into recursive self-improvement before 2027."

You play as a researcher, research manager, CEO or government, in a game for 1 to 4 players that runs in your browser. The game is fiction. Its footer says it is inspired by the AI 2027 scenario, and that all of its labs, people and events are made up. It does not tell you what real labs will do. What it gives you is a way to sit with the question this article asks: if a loop like this were close, who could slow it, and how?

If you want to go deeper on the real research first, the AI safety self-study path lists where to start.

Frequently asked questions

Is recursive self-improvement happening today?

Researchers have built pieces of it, such as programs that improve their own scaffolding, but Wikipedia still describes the full loop and the explosion it might cause as theoretical. Each early system relies on a fixed model or on automated scores that many tasks lack.

Is recursive self-improvement the same as an intelligence explosion?

No. Recursive self-improvement is the loop. An intelligence explosion is what some people think the loop could cause if each round came faster than the last.

Who first described the idea?

I. J. Good set out the core argument in 1965, and Eliezer Yudkowsky later coined the term "seed AI" for the starting system.

Why would a self-improving AI be hard to control?

Its goals could drift or be misread, it might treat being shut down as an obstacle, and controls like a kill switch get weaker as the system gets smarter. Others argue the loop would stall long before that point.

Get started

Want to see how the race feels from the inside? Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional. You will need a recent Chrome, Edge, Firefox or Safari, because it runs on WebGL 2.