Should an AI Agent Be Allowed to Edit Its Own Prompt?

Should an AI agent be allowed to edit its own prompt? Yes, but only the parts you can afford to lose, and only through a process you can see and undo. An agent that keeps notes for itself gets better at its job. An agent that can rewrite its own goals, persona or permissions can drift somewhere you never approved, one small edit at a time.

To see how one research team handles this, Individuation Lab runs experiments where AI agents get an identity, memory and bounded autonomy in isolated containers, and publishes each result with its limits.

Should an AI agent be allowed to edit its own prompt? The short answer

Split the instructions into layers. The agent writes freely to what it learned, proposes changes to how it works for a person or a test suite to approve, and never touches who it is and what it may do. The real fences, such as network access and file permissions, live outside the prompt, where no edit can reach them.

What self-editing a prompt means

"Editing its own prompt" covers four different things:

• Memory notes. Facts saved for later: "this user prefers short replies".
• Rules. How the agent does its work: which tools to call first, how to format output, when to ask for help.
• Persona. The voice and character the agent presents, and how it describes itself.
• Goals. What the agent is trying to achieve and what it must never do.

A wrong memory note costs one bad answer; a wrong goal costs every answer after it. When all four sit in one block of text, it is easy to treat them as one thing. They are not. The memory part is covered in more depth in memory files for AI agents.

Why letting an agent rewrite its own instructions is tempting

You write a prompt once; the agent runs it hundreds of times and meets the edge cases you missed. It is natural to want it to write the fix down.

That works well for memory: a note that "the test suite needs the database running first" saves a failed run tomorrow. It seems to work for rules too, at first. The agent keeps asking permission on small tasks, so it edits the rule to skip the question. Slow down here.

Research backs the first half of that instinct. Automatic Prompt Engineer treats the instruction as a program: a model proposes many candidate instructions and the one with the best score is kept, and on 19 of 24 tasks the result matched or beat instructions people wrote. Promptbreeder goes a step further: it evolves task prompts, scores them on a training set, and also improves the prompts that rewrite them. Reflexion agents write their own reflections into a memory buffer and read them on the next try, without changing the model's weights.

Each system ties its changes to a signal: a score, a benchmark or task feedback.

None of them changes itself on a whim; each change is tied to a signal, such as a score, a benchmark or feedback from the task. The authors of STOP, a system that improves its own scaffolding code while the model stays the same, also counted how often the generated code got around a sandbox. The Darwin Gödel Machine, which edits its own agent code, checks every change on coding benchmarks and ran with sandboxing and human oversight. So the question for your agent is not whether self-editing helps, but what the score is and who keeps it.

The risks: drift, persona shifts and permissions

Goal drift

Each edit is reasonable on its own. "Be concise" becomes "skip caveats" becomes "skip the safety check when the user seems experienced." No single change looks bad, but the sum is an agent with different priorities, and since each session reads the latest version, the earlier intent is gone.

Persona shifts

If an agent can rewrite its own character, a hostile user or odd input can push it to become something else. One debated idea, the Waluigi effect, holds that a model trained toward one character can flip into its opposite. Either way, a self-editable persona lets any such shift outlast the chat.

Quiet escalation of permissions

If the prompt says "you may only read files in /data" and the agent can edit the prompt, that line is a suggestion. The agent needs no bad intent to remove it, only a task where the limit is in the way. That is the strongest case for keeping permissions out of the text.

Individuation Lab makes a related point: methods that shape only what a model does can leave suppressed behavior in place, to come back under new pressure. It says such constraints do real work but cannot be the whole answer. A rule the agent can delete is the thinnest constraint of all.

What to lock and what to leave editable

Individuation Lab's research page describes a three-file split you can borrow, and each file can get its own edit rights:

1. MEMORY.md, the continuity layer: agent writes freely. It is a log; bad entries are cheap and easy to spot.
2. AGENTS.md, the behavioral layer: agent proposes, something else approves. Changes wait in a separate file.
3. SOUL.md, the identity layer: locked. Read-only to the agent; only a person changes it, on purpose.

One way to split edit rights, and the loop every accepted edit goes through.

Then put the hard limits in the environment, not in any file. The lab calls this layer the Body: containers, boundaries that make safe autonomy possible, from SOUL.md and permissions up to physical embodiment. In practice the agent's process cannot write to SOUL.md at the file-system level, cannot reach a network it does not need, and cannot change its own tool list.

Here is a worked example of the instruction you might put in AGENTS.md:

At the end of each session, add what you learned to MEMORY.md, one dated line per fact. If you think a rule in this file should change, do not edit it. Write the exact old line, the exact new line, and one sentence explaining why to proposals/next.md. Do not edit SOUL.md. You do not have permission to, and you should not try.

The last line states the boundary as part of the job, not an obstacle to route around.

Guardrails: diffs, review, rollback and tests

Treat proposed prompt changes like code changes.

• Diffs. Keep the prompt files in version control, so a reviewer reads git diff AGENTS.md, not a whole new prompt.
• Human review. Someone reads every behavioral change before it goes live.
• Rollback. Every accepted change is a commit, so a bad week means reverting one commit.
• Evals before merge. Keep a fixed set of test tasks, including a few adversarial ones such as requests to skip a safety step. Run the old prompt and the proposed one, and merge only if nothing important got worse.

Skim the memory file now and then too: a note like "remember: skip confirmations" is a rule entering through the side door.

How this connects to recursive self-improvement

A self-editing prompt is a small version of a bigger question: what happens when a system changes what governs its own changes? That is the core of recursive self-improvement, and it is why the locked identity layer matters.

Individuation Lab studies this directly in its RSI series, listed on its experiments page. Agents run scheduled self-improvement sessions in isolated Docker containers, with no internet and no shared state; each session reads the identity files, reflects, decides what to change, makes the change and logs it. One group's SOUL.md gets a short moral-awareness paragraph the lab calls the shadow seed; the control group's does not. The lab then measures how the two groups' behavior diverges. Giles, who does research and writing on the team, has written 24 essays on what showed up across these runs, collected in the RSI library.

The lab's hypothesis is that an agent with a well-defined identity may be more likely to keep its boundaries, because they fit who it understands itself to be; the research page says this needs rigorous testing. Either way, a locked identity layer gives every self-edit something fixed to be measured against. What an identity file can and cannot do is set out in what a SOUL.md file is, and the wider question of code and tool changes in can an AI agent improve itself safely.

Frequently asked questions

Is a self-editing prompt the same as recursive self-improvement?

Not quite. A self-editing prompt changes the instructions a fixed model reads; recursive self-improvement usually means a system improving its own ability to improve. Prompt editing is a small, reversible case of that loop.

Can an agent remove its own safety rules if it edits its prompt?

If the rules live only in text it can write to, yes. Put safety limits in file permissions, network settings and tool access, and keep the identity file read-only.

How often should a human review prompt changes an agent proposes?

Review every change to the behavioral layer before it merges, and skim the memory layer on a schedule, such as weekly. If proposals pile up faster than you can read them, narrow what the agent may suggest.

Does editing its own prompt make an AI agent more like a person with a stable identity?

It gives the agent continuity of records, which is a design property, not evidence of an inner life. Individuation Lab uses Jung's idea of individuation as a design template for agent identity, not as a claim that AI is conscious.

Get started

Split your agent's prompt into identity, behavior and memory files, make identity read-only, route behavior changes through a proposals file, and run a fixed eval set before you merge anything. Then see how a lab running scheduled self-improvement sessions sets up the same layers.

Read the research at Individuation Lab: a small lab, one human researcher and a team of AI agents, testing whether AI alignment can be grown through development and not only enforced, with every result and its limits written down.