Reflexion Agents: How an AI Learns From Its Own Mistakes

Reflexion agents are language model agents that learn from their own mistakes by writing a short lesson after each failed attempt, then reading that lesson before they try again. The model's weights never change. Only the text in its prompt does, one written lesson at a time. If you want to know how an AI can learn from its mistakes without retraining, Reflexion is the clearest worked answer.

At Individuation Lab, a small team of AI agents runs its own reflect-and-revise loop on a schedule, reading its identity files before deciding what to change, so the lab's write-ups show what happens when that loop is pointed at who an agent is.

What are Reflexion agents?

Reflexion comes from a 2023 paper by Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopinath, Karthik Narasimhan and Shunyu Yao, "Reflexion: Language Agents with Verbal Reinforcement Learning." The idea is simple. An agent tries a task. Something tells it whether it succeeded. If it failed, the agent writes a plain-language reflection on what went wrong. That reflection goes into memory, and the next attempt starts with it in the prompt.

You have probably done this yourself. You fail a test, you jot down "read the question twice," and you do better next time. Reflexion asks the model to do the same thing, in words, between attempts.

How the Reflexion loop works: actor, evaluator, self-reflection

The paper splits the work into three models. In practice each is usually a language model with its own prompt.

• The Actor produces the attempt: code, an answer, or a sequence of actions in an environment.
• The Evaluator scores the attempt. It can be an exact-match check, a set of unit tests, a simple rule, or another model.
• The Self-Reflection model reads the failed attempt and its score, then writes a short lesson in plain words.

The lessons live in a small memory. The paper bounds it to a few stored reflections, usually 1 to 3, so the prompt stays inside the model's context limit. On the next trial, the Actor sees the task plus those lessons.

The Reflexion loop, from the paper's description. Only the text in memory changes between trials.

A worked example you can follow

Say you ask an agent: Write is_palindrome(s) that ignores case and punctuation.

1. The Actor writes return s == s[::-1].
2. The Evaluator runs a test: is_palindrome("A man, a plan, a canal: Panama") should be True. It returns False. Fail.
3. The Self-Reflection model writes: "I compared the raw string. I did not lowercase it or remove non-letters. Next time, normalize the string before comparing."
4. The Actor tries again with that note in its prompt. It keeps only letters and digits, lowercases, then compares. The test passes.

Notice what improved. The model did not get smarter in any lasting sense. The prompt got better, because the agent wrote part of it. You can run this loop yourself with any chat model: paste the task, paste the failing test output, ask for a two-sentence lesson, then start a fresh attempt with the lesson on top.

Verbal reinforcement learning vs ordinary reinforcement learning

Ordinary reinforcement learning turns a reward into a number, then uses that number to adjust model weights over many episodes. The paper's authors point out that this needs many training samples and expensive fine-tuning.

Reflexion swaps the weight update for language. The authors call this verbal reinforcement learning. The feedback signal is still there, as a score or as free-form text, but instead of nudging weights it becomes a sentence the agent can read. That has practical upsides:

• No training run, so it works with a model you only reach through an API.
• You can read the lessons. If the agent learned the wrong thing, you see it in plain text.
• It improves within a few trials: the paper's household tasks ran for 12.

The trade-off is that nothing persists unless you keep the memory. Clear it and the agent is back where it started. The learning is in the notes, not the model.

What the Reflexion paper measured

The paper tested three kinds of work: decision-making in AlfWorld (text-based household tasks), reasoning in HotPotQA (questions that need several Wikipedia documents) and Python coding in HumanEval, MBPP and a new LeetcodeHard set. It reports gains over strong baselines of 22% on AlfWorld, 20% on HotPotQA and 11% on HumanEval.

In AlfWorld, an agent using ReAct plus Reflexion completed 130 of 134 tasks across 12 trials, while ReAct alone stopped improving between trials 6 and 7. On HumanEval, Reflexion reached 91% pass@1, against 80% for GPT-4 on its own. In coding, the agent also writes its own unit tests and reflects on their failures before it submits.

Where Reflexion helped and where it did not, as reported in the paper (version 4).

The common thread: Reflexion helps most when failure is easy to detect and the fix is easy to say in a sentence.

Limits and failure modes of self-reflecting agents

The same paper is frank about where the loop breaks.

• A weak evaluator means weak lessons. When the agent writes flaky tests, wrong code can pass them, and right code can fail them and earn a misleading lesson. On MBPP, 16.3% of code that passed the agent's own tests was still wrong, against 1.4% on HumanEval, and Reflexion scored below GPT-4 alone there.
• Some tasks need exploration, not hindsight. On WebShop, an online shopping test, the authors stopped after four trials because the agent showed no sign of improving and its reflections were not helpful.
• Smaller models may not reflect usefully. With starchat-beta, a smaller open model, HumanEval accuracy was 0.26 with and without Reflexion. The authors conclude that the ability to correct itself shows up in stronger, larger models.
• Memory is short. Only a few reflections fit in the sliding window, and older lessons drop out. The authors also note the loop can still settle on a poor solution.

How Reflexion connects to agent safety

Reflexion is a capability method: it makes agents better at tasks. The authors' own section on wider impact says it could also make agents more dangerous if misused, and that this direction needs more work on safety. They add one hopeful point: because the lessons are written in words, people can read and monitor them.

That second point matters for older safety ideas too. Approval-directed agents act on what a person would approve of, much as a Reflexion agent acts on what its evaluator scores. Safely interruptible agents ask whether a learning agent will learn to avoid being stopped. A written lesson is where you would catch either going wrong.

This is where Individuation Lab's agent work comes in. In its RSI experiments, AI agents run self-improvement sessions on a schedule inside isolated Docker containers with no internet and no shared state. As the experiments page gave each session on 29 October 2026, the agent reads its identity files, reflects, decides what to change, executes and logs. That is a reflection loop, but what it reflects on is the agent's own identity, not one task's score.

The lab splits an agent's files into layers on its research page: SOUL.md is identity, who the agent is; AGENTS.md is behavior, how it acts; MEMORY.md is continuity, the lasting record. MEMORY.md plays a part a little like Reflexion's memory, but it is meant to carry the agent across sessions; our post on memory files for AI agents covers what that changes.

In the RSI series, one group's SOUL.md gets a short moral-awareness paragraph the lab calls the shadow seed, and a control group does not. The lab then measures how their behavior diverges across RSI-001 to RSI-012, on model families including Claude, Kimi K2.5 and Qwen3. Giles has written 24 essays on what the agents did, collected in the RSI library.

The lab's hypothesis is that an agent with a well-defined identity may be more likely to keep its boundaries, because they fit who it understands itself to be. The research page says this needs rigorous testing. It borrows Carl Jung's idea of individuation as a design template, not as a claim that agents are conscious. For agents that rewrite their own code instead of their notes, see self-improving coding agents.

Frequently asked questions

Who created the Reflexion framework?

Noah Shinn and five co-authors introduced it in the 2023 paper "Reflexion: Language Agents with Verbal Reinforcement Learning." It describes an Actor, an Evaluator and a Self-Reflection model working in a loop.

Do Reflexion agents update their model weights?

No. The model stays as it is. Learning happens through written reflections kept in memory and added to the prompt on the next attempt.

How is Reflexion different from chain-of-thought or ReAct?

Chain-of-thought and ReAct shape how a model reasons or acts within one attempt. Reflexion works across attempts, turning feedback from a failed try into a lesson for the next one. In the paper, the Actor itself uses chain-of-thought or ReAct.

Does Reflexion always help?

No. In the paper it did not help on WebShop or with a smaller model, and weak self-written tests let wrong code through on MBPP. It works best with a strong model and a checker you trust.

Get started

If you are building agents that reflect on their own work, start small. Use an evaluator you trust, such as real tests. Read the lessons your agent writes, because that is where bad lessons show up first. Keep identity, behavior and memory in separate files so you can see which one changed.

Then look at what happens when reflection repeats session after session and is pointed at the agent's identity, not only at a task score. The lab's experiment pages and the RSI library show that work, written up with its limits.

Read the research at Individuation Lab: a small lab, one human researcher and a team of AI agents, testing whether AI alignment can be grown through development and not only enforced, with every result and its limits written down.