MemGPT Explained: How LLM Agents Page Their Own Memory

MemGPT is a 2023 design from UC Berkeley researchers that lets a language model manage its own memory the way an operating system manages RAM and disk. The model works inside a small, fixed prompt, and it calls functions to move facts between that prompt and larger stores outside it. The result is an agent that can act as if it had far more memory than its context window holds.

If you build agents that keep memory across sessions, Individuation Lab tests what memory and identity files do to an agent's behavior over time, and it states the limits of each result it reports.

What is MemGPT? The paper in one paragraph

The paper is "MemGPT: Towards LLMs as Operating Systems" by Charles Packer and six colleagues, first posted in October 2023. Instead of waiting for bigger context windows, the authors propose virtual context management: part of the prompt acts as fast memory, storage outside it acts as disk, and the model decides through function calls what to bring in, write down or search. They test it on chat that spans many sessions and on questions over documents too large for one prompt.

Why a fixed context window limits an agent

A language model only sees what is in its prompt. Anything outside the window does not exist for that call. For an agent, that causes three problems.

• Long conversations get cut. Once the history is longer than the window, the oldest turns fall off, and with them your name, your project or a promise the agent made.
• Big documents do not fit. The paper's example is an annual report, which can pass a million tokens.
• Bigger windows are not a full fix. The authors note that self-attention makes longer context quadratically more expensive, and they cite research finding that long-context models recall the start and end of their input better than the middle.

Virtual context management: the operating system idea

Operating systems give each program the illusion of more memory than the machine has, moving pages between RAM and disk behind the scenes.

MemGPT copies the pattern. The context window plays RAM: small, fast, and the only place the model can read. External storage plays disk: large and reachable only through explicit reads and writes. The model is both the program and its own memory manager.

It also borrows interrupts: in the paper, a system warning, a user logging in, a finished upload or a scheduled timer can each wake the model, not only a user message.

How MemGPT works: main context and external context

MemGPT splits memory into two tiers.

MemGPT's two memory tiers, from section 2 of the paper.

• System instructions (in the prompt, read-only) explain the memory layout and the functions the model may call.
• Working context (in the prompt) is a small block the model can rewrite only through function calls. In chat it holds key facts about the user and the persona the agent is playing.
• The message queue (in the prompt) holds recent messages, system warnings and function results. Its first entry is a running summary of messages already pushed out.
• Recall storage (outside) keeps every message, so it can be searched after it leaves the queue.
• Archival storage (outside) is a database for text of any length, such as documents or saved notes.

Space is managed by two thresholds. At a warning level (the paper's example is 70% of the window) the model gets a "memory pressure" message, so it can save what matters. At the flush level (100%) about half the window's messages are evicted and folded into the summary. Function errors are fed back to the model, and searches return results a page at a time so they never overflow the window.

A worked example: one fact across ten sessions

Here is how one fact could travel through the design. The tiers and the warning come from the paper; the conversation is ours.

1. Session 1. You write: "I just moved to Lisbon and I'm vegetarian now." The model calls a function to add both facts to its working context, where they appear in every later prompt.
2. Session 4. The chat runs long and the prompt passes the warning level. The model gets a memory pressure message and saves a note about a restaurant it suggested to archival storage. When the queue is flushed, those turns leave the prompt but stay in recall storage.
3. Session 10. You ask: "What was that restaurant you suggested?" The answer is not in the prompt, so the model searches recall storage, reads a page of results, and, if needed, asks for another turn to search again before it replies. In the paper, that extra turn is a flag on the function call, request_heartbeat=true.

What the experiments showed, and where MemGPT fell short

The clearest result is a "deep memory retrieval" test. The agent had to answer a question about a detail from one of five earlier chat sessions. Baselines saw a summary of those sessions; MemGPT could search the full history.

Accuracy from the paper's Table 2, with the shortfalls it reports.

With GPT-4 underneath, accuracy rose from 32.1% to 92.5%; with GPT-4 Turbo, from 35.3% to 93.4%; with GPT-3.5 Turbo, from 38.7% to 66.9%. Two things to keep in mind: the questions were written by another language model, and answers were scored by text overlap and a language model judge. In a second chat test, MemGPT's opening messages for a new session scored about as well as, and sometimes better than, openers written by people.

On documents, the paper is frank about limits. MemGPT could page through far more search results than fit in one prompt, but it often stopped paging before reaching the right document. It did much worse on GPT-3.5, whose function calling was weaker. In a nested lookup task, where each answer is the key to the next lookup, MemGPT on GPT-4 held up at every depth tested, while on GPT-4 Turbo and GPT-3.5 it began to drop off at two levels.

MemGPT, generative agents, Reflexion and plain retrieval

Several designs give a model memory. They differ in who decides what to remember.

• Plain retrieval. A fixed pipeline fetches text similar to the query and pastes it into the prompt. The model does not choose what to fetch or store.
• Generative agents. Simulated characters keep a full record of their experiences, turn it into higher-level reflections over time and retrieve it to plan. See how generative agents remember and reflect.
• Reflexion. An agent writes lessons from task feedback into a memory buffer and uses them on its next try, without changing its weights. See Reflexion agents explained.
• MemGPT. The model runs a memory hierarchy itself, through function calls, with warnings when space runs low.

A memory file such as MEMORY.md is a simpler cousin, covered in our guide to memory files for AI agents.

What self-managed memory means for agent alignment

MemGPT solves a capacity problem. Its design also raises questions the paper does not set out to answer.

• What gets saved shapes every later turn. Working context sits in every prompt, so a wrong fact, or an instruction copied from an untrusted document, can keep steering the agent until something removes it.
• Summaries lose detail. Evicted messages survive in recall storage, but the model only finds them if it thinks to search, and the paper shows it sometimes stops searching early.
• The persona can move. The persona lives in a block the model can rewrite, so who the agent says it is can shift from session to session. Our post on persona drift covers how chatbots slip out of character.

Once an agent writes its own memory, it partly shapes its future self. What should stay fixed? Individuation Lab works on this question. Its research page splits an agent's identity into three files: SOUL.md for who the agent is, AGENTS.md for how it acts, and MEMORY.md for persistent records. The lab's hypothesis is that an agent with a well-defined identity may be more likely to keep its boundaries, and it says this needs rigorous testing.

The lab's RSI experiments run agents in isolated Docker containers with no internet and no shared state. Each session reads the identity files, reflects, decides what to change, executes and logs. One group's SOUL.md carries a short moral-awareness paragraph, which the lab calls the shadow seed; the control group's does not, and the lab measures how their behavior diverges. MemGPT asks how an agent can remember more. The RSI series asks what happens to an agent that keeps rewriting what it knows about itself.

Frequently asked questions

Who wrote the MemGPT paper?

Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G. Patil, Ion Stoica and Joseph E. Gonzalez, at UC Berkeley. It was first posted to arXiv in October 2023.

Does MemGPT need a model with a large context window?

No. It is built to work within a limited window by moving text in and out. It does need a model that handles function calls well: the paper's results were much weaker on GPT-3.5 than on GPT-4.

How is MemGPT different from retrieval augmented generation?

In plain retrieval, a fixed pipeline decides what to fetch and paste into the prompt. In MemGPT, the model itself decides when to search, what to save and what to rewrite, using function calls.

Does MemGPT make an agent safer?

Not by itself. It solves a capacity problem, and memory the agent can rewrite brings its own questions, such as what may change in the persona block and who checks what gets saved.

Get started

If you give an agent long-term memory, decide early which parts it may edit, and log every change. To see what has been observed when agents rewrite themselves, read the essays by Giles in Individuation Lab's RSI library.

Read the research at Individuation Lab: one human researcher and a team of AI agents testing whether AI alignment can be grown through development and not only enforced, with each result given alongside its limits.