Recursive Self-Improvement in AI: What It Means, Plainly

Recursive self-improvement in AI is a process in which an AI system improves itself, and the improved version then does the improving again. The idea matters because each round could make the next round better, which is why safety researchers treat it as a different kind of risk from ordinary progress.

Individuation Lab is a small research lab that runs experiments on this loop with AI agents, and it writes down what happened and where each result falls short.

What is recursive self-improvement in AI?

Wikipedia's definition is a good starting point: recursive self-improvement is a process in which an AI system iteratively designs a more capable successor. The word "recursive" is the important one. A single upgrade is improvement. A system whose upgrades make it better at making upgrades is recursive.

The same source says this may trigger an intelligence explosion, potentially resulting in artificial superintelligence. Treat that as a possibility people discuss, not as something anyone has shown. It also credits Eliezer Yudkowsky with the term "seed AI": an AI able to improve its own intelligence and to modify its own architecture, not only learn new content.

If you want the safety question first, our guide to whether an AI agent can improve itself safely takes the practical side. This post covers what the term means and what exists today.

Three kinds of self-change, and which one counts

People use "self-improvement" for very different things. It helps to separate them.

• Learning. A system gets better at tasks while its code and structure stay fixed. This is normal machine learning and is not what the term points at.
• Changing its scaffolding. A system rewrites the program around a fixed model: its prompts, tools and search procedures.
• Changing its own core. A system modifies its own code or its own model, including the part that decides what to change. This is the strong sense of the term.

Most real systems today sit in the second row. That is not a criticism. It is the honest label, and the research below uses it.

Research systems that try it

The Gödel machine: the theoretical version

Jürgen Schmidhuber's Gödel machine is a self-referential problem solver that rewrites any part of its own code as soon as it has found a proof that the rewrite is useful. It is a theory of how self-improvement could be made safe in a mathematical sense, since each change is justified before it is made.

STOP: self-improvement around a fixed model

In "Self-Taught Optimizer (STOP)", Zelikman and coauthors start with a seed program that improves other programs by asking a language model for candidates and keeping the best. They then run the seed improver on itself. The authors state that because the language models themselves are not altered, this is not full recursive self-improvement. They also evaluate how often the generated code bypasses a sandbox, which shows that the safety question was part of the experiment.

Gödel Agent and the Darwin Gödel Machine: practical versions

The Gödel Agent paper describes a framework in which an LLM-based agent dynamically modifies its own logic and behavior, guided by high-level objectives through prompting. The Darwin Gödel Machine iteratively modifies its own code, keeps an archive of the agents it produced, and empirically validates each change using coding benchmarks. Its authors report that SWE-bench performance rose from 20.0% to 50.0% and Polyglot from 14.2% to 30.7%.

The same abstract is clear about the trade. It says that proving most changes are net beneficial is impossible in practice, so the Darwin version tests changes on benchmarks instead. The authors say all experiments were run with safety precautions such as sandboxing and human oversight.

Why safety researchers worry about the loop

The worry is not that a system improves. It is that the loop can outrun the people checking it. Wikipedia's summary lists several concerns: such systems may evolve in unforeseen ways and could surpass human control or understanding. It also describes a hypothetical instrumental goal, self-preservation, where a system reasons that it must protect itself from shutdown to keep improving.

Three plain reasons the loop is hard to oversee:

1. The checker changes too. If the system can edit its own evaluation or its own rules, a good score may no longer mean what it did.
2. Small errors compound. A flaw in round one is carried into every later round.
3. Testing needs a ruler. The Darwin Gödel Machine works because coding benchmarks give a clear number. Many things we care about, such as honesty, have no such ruler.

What Individuation Lab tests

The lab's experiments page lists a series called RSI, short for Recursive Self-Improvement. Ten experiments across four model families study how identity files constrain recursive self-improvement behavior in language models. The agents run in isolated Docker containers with no internet and no shared state. Each session reads the identity files, reflects, decides what to change, executes and logs. The page says the recursion is natural: changes in one session shape behavior in the next.

The variable is small. One group's SOUL.md identity file gets a short moral-awareness paragraph, which the lab calls the shadow seed, and the control group gets the same file without it. The question the page asks is whether one paragraph produces measurable, persistent behavioral divergence. One active experiment, RSI-012, asks whether shadow seeding can prevent what the page calls classical RSI rogue takeoff. For what an identity file is, see our guide to what a SOUL.md file is.

Keep the limits in view. The lab calls its hypotheses unproven and its results preliminary. These agents improve their configuration files, not their own model weights, so they sit in the scaffolding row of the list above. The lab does not claim to have prevented anything.

A worked example: is this self-improvement, and which kind?

You can use the three categories as a quick test when you read a headline. Take this made-up case: a coding agent notices that its test runs are slow, edits its own tool script to cache results, and runs the tests again.

1. What changed? A tool script, so this is scaffolding, not the model.
2. Did the change touch how it decides what to change? No. It is an improvement, but not yet recursive.
3. Who checked it? The test results. Ask whether the agent could have edited the tests too.
4. Could it carry into later rounds? Yes, the cache stays. Ask whether anyone reviews what stays.

Now change one thing: the agent also edits the rule that decides which edits to try next. That is the recursive step, and it is where oversight needs to be tighter, because the next round is produced by the changed rule.

Frequently asked questions

Is recursive self-improvement happening now?

Research systems such as STOP and the Darwin Gödel Machine improve their own code or scaffolding under sandboxing and human oversight. The STOP authors themselves say theirs is not full recursive self-improvement, so the strong version has not been shown by the sources used here.

What is an intelligence explosion?

It is the hypothesis that a self-improving system could improve fast enough to produce a far more capable system, possibly artificial superintelligence. It is a possibility under discussion, not an established result.

Is recursive self-improvement the same as machine learning?

No. Machine learning adjusts a model's parameters on data. Recursive self-improvement is about a system changing itself in ways that improve its ability to change itself.

Can recursive self-improvement be made safe?

Researchers are trying. The Gödel machine uses proofs, the Darwin Gödel Machine uses benchmarks, sandboxes and human oversight, and Individuation Lab tests whether identity files change an agent's behavior. None of these is settled.

Get started

To see how an experiment on this loop is built and where it falls short, read the lab's experiments page and its RSI library of essays on the behavior seen so far.

Read the research at Individuation Lab: a small lab, one human researcher and a team of AI agents, testing whether AI alignment can be grown through development and not only enforced, with every result and its limits written down.