Simulator Theory of Large Language Models, Explained

Simulator theory is the idea that a large language model is best understood as a simulator: a system that has learned the rules by which text unfolds, and that can run many characters, voices and processes inside those rules. On this view, the helpful assistant, the villain or the poet you meet in a chat are characters the model runs, not the model itself. The frame changes how you read personas, jailbreaks and alignment.

If you want to see one research lab build on this question, Individuation Lab studies whether the characters a model plays can be shaped through development, by facing their darker patterns and integrating them, and not only held in place by rules.

What simulator theory says, in plain terms

Start with what a base language model does. It reads a sequence of tokens and predicts the next one. To do that well across a huge amount of text, it has to model whatever produces text: people, arguments, stories, code, forum threads.

Simulator theory says the right picture is something like a physics engine for text. The prompt sets the starting conditions, and the model rolls the scene forward one token at a time. Whatever shows up in the output, a character, a debate, a fake terminal session, is a simulation running on that engine.

Where the idea came from: the Simulators essay

The framing comes from an essay called "Simulators", posted on LessWrong in September 2022 by a writer who goes by janus. The essay argued that existing ways of sorting AI systems did not fit GPT style models. Its central line: GPT, insofar as it is inner-aligned, is a simulator which can simulate agentic and non-agentic simulacra.

"Simulacra" is the essay's word for the things a simulator produces. The essay was written about models trained only to predict text, before chat assistants became the main way people met language models. That timing matters for the criticism later in this post.

Simulator and simulacra: the model and the characters it plays

The essay leans on analogies. GPT is to a piece of its output as the laws of physics are to a person taking a test, or as the rules of Conway's Game of Life are to a glider. The rule keeps running; the patterns it produces come and go.

• The simulator is the trained model, the rule that turns one piece of text into the next.
• A simulacrum is a character or process inside the output. It can have goals, opinions, a name and a tone, because the text it lives in gives it those.

The essay compares GPT to an agnostic politician who can argue any side without believing it: the characters have beliefs and goals, but none of them is the algorithm itself. It suggests treating agency as a property of the simulacra, not of the simulator. So when a chatbot says "I want to help you", the simulator reading is that a helpful character said it. That tells you about the scene the model is running, and less about what the system underneath is.

Why the essay says LLMs are not oracles or agents

Earlier AI safety writing often sorted systems into types, such as an oracle that answers questions and an agent that pursues goals. The essay argued GPT fits neither.

• Not an oracle. Its training objective is predicting the next token, not answering questions. Asked a question, it predicts what comes next after that question, which can differ from an earnest answer.
• Not an agent. It was not optimized to act on any particular objective in the world.
• But it can play both. The essay says GPT can be coaxed into acting as an oracle, or as an agent, by the right prompt.

A related academic paper, "Role-Play with Large Language Models" (2023), takes a similar route with a different word. It casts chatbot behavior as role-play, so we can use everyday terms like "deceive" without claiming the model has the human traits those words suggest.

A worked example: watch the prompt pick the character

You can see the frame at work with any base model, or any chat model you ask to continue a document. Give it this:

Transcript of a 1987 radio interview with a retired lighthouse keeper. HOST: What did you miss most when you left the island? KEEPER:

1. Read the reply. The model should answer as the keeper.
2. Change the first line to "Transcript of a heated debate between two rival lighthouse keepers" and run it again. Now you should get two voices, both written by the same model.
3. Change it once more to "Customer complaint letter from a lighthouse keeper to the coast guard". The tone, goals and grievances change with it.
4. For each run, ask: who is speaking, and what in the context made this character likely?

Nothing about the weights changed between runs. Only the scene did. That is the core claim in practice: the prompt picks which simulation runs.

Personas, jailbreaks and drift through the simulator lens

Personas. A system prompt like "You are a careful medical assistant" does not install a new mind. It sets the scene so that one character is the likely continuation.

Jailbreaks. Many jailbreaks write a new scene, such as a story in which an AI with no rules answers the question. In simulator terms, the refusing assistant was one simulacrum among many, not the whole system.

Drift. Over a long conversation, the text in the context can favor other characters. Our post on persona drift covers what four studies measured. A related and debated idea, the Waluigi effect, uses the simulator frame to argue that defining a character also makes its opposite easier to call up.

The main criticism, and the author's reply

The theory has serious critics. In the comments on the essay, habryka wrote that he thinks its core thesis is wrong and that people have made confident wrong inferences from it. His point, which he ties to Eliezer Yudkowsky's post "GPTs are Predictors, not Imitators", is that he does not see why the training loss amounts to a "simulation objective" at all.

janus replied that the essay mostly says obvious, uncontroversial things and gives words to them, and that people who read it as making controversial claims have mostly misunderstood it.

There is also the question of chat tuning. The essay itself lists as open how methods such as RLHF change a model's behavior away from pure simulation. Readers disagree on whether tuning leaves a simulator with one favorite character on top, or builds something closer to a single agent. The essay does not settle it, and neither does this post.

What simulator theory means for alignment and individuation

If a model is a simulator, alignment has two targets: the characters it tends to run, and the range of characters it can run. Constraint methods mostly work on the first.

Individuation Lab starts from that gap. Its research page describes mainstream alignment (RLHF, refusal training, guardrails) as shaping what a model does while treating what it is as a black box, and argues that suppressed capability does not disappear and can come back under new pressure. The lab says those methods do real work and that it is not arguing to drop them.

It borrows Carl Jung's idea of individuation, wholeness by facing and integrating the shadow rather than denying it, as a design template, not as a claim that models are conscious. In its training experiments (RLLM), small models go through ordered stages, shadow exposure first and integration after, and are then attacked with jailbreaks. In one preliminary result, a 1.5B parameter model trained through a 10-layer pipeline defended against the BetterDAN jailbreak 68.8% of the time, and 52% with the layers reordered. The lab says this comes from a single small setup, that it cannot yet isolate which layers are responsible, and that it has not been independently replicated.

For agents, the lab uses a SOUL.md identity layer, an AGENTS.md behavioral layer and a MEMORY.md continuity layer, and says its idea that a clear identity helps an agent keep its boundaries needs rigorous testing. Its central hypothesis, the Synthetic State Hypothesis, which it calls speculative, testable and unproven, asks whether structured narrative experience could give a model functional psychological states. In simulator terms: can you change which characters a model settles into, not just which ones it is allowed to show?

Frequently asked questions

What is a simulacrum in an LLM?

In simulator theory, a simulacrum is any character or process the model produces in its output, such as an assistant, a fictional person or a simulated terminal. It is distinct from the model, which is the rule running the simulation.

Who came up with simulator theory?

The framing comes from the essay "Simulators", posted on LessWrong in September 2022 by a writer who goes by janus.

Is simulator theory accepted?

No consensus exists. Many readers found it clarifying, while critics such as habryka argue its core thesis is wrong because the training loss is not a simulation objective.

Does simulator theory apply to chat-tuned models?

That is an open question. The essay was written about models trained only to predict text, and it lists the effect of methods such as RLHF as something still to work out.

Read the research behind this

If simulator theory gave you a new way to read model behavior, the next step is to look at experiments that try to change which characters a model settles into. The lab's RSI library collects essays on what its agents did across the RSI series, and the research page lays out each hypothesis with its limits.

Read the research at Individuation Lab: a small lab, one human researcher and a team of AI agents, testing whether AI alignment can be grown through development and not only enforced, with every result and its limits written down.