Voyager AI Agent: How a Skill Library Lets an Agent Grow

The Voyager AI agent is a 2023 research system that plays Minecraft by writing code. When a piece of code works and a checker confirms it, Voyager saves it to a skill library and calls it again in later code. The language model underneath never changes. What grows is a library of tested programs, which is why the agent keeps getting more capable without any retraining.

Individuation Lab studies the other half of that story: agents that change themselves on a schedule, and whether their character can grow along with their abilities. Its write-ups are a good next read once you know how Voyager works.

What is the Voyager AI agent?

Voyager comes from the paper "Voyager: An Open-Ended Embodied Agent with Large Language Models" by Guanzhi Wang and seven coauthors, posted in May 2023. The agent explores Minecraft with no fixed goal beyond finding new things. It calls GPT-4 as a black box, with no fine-tuning, and acts through Mineflayer's JavaScript API, so every action it takes is a function it wrote.

Most agents start each task cold: they plan, act and forget. Voyager keeps what worked. A skill learned early, such as mining wood, becomes a function that a later skill, such as crafting a pickaxe, can call. The authors call the skills temporally extended, interpretable and compositional, and say this lets abilities compound while avoiding the forgetting seen in other lifelong learning methods.

One turn of Voyager's loop, from the paper's method section.

The three parts: curriculum, skill library and iterative prompting

• Automatic curriculum. GPT-4 reads the agent's inventory, surroundings, finished and failed tasks, and proposes the next one, such as "Mine 3 iron ore". It adapts to where the agent is: in a desert it might learn to harvest sand and cactus before iron.
• Skill library. Each skill that passes is stored as a program. Its key is an embedding of a short description of what it does, written by GPT-3.5.
• Iterative prompting. Voyager runs its code and reads back three kinds of feedback: game messages ("I cannot make an iron chestplate because I need: 7 more iron ingots"), execution errors (it once tried to craft an acacia axe, which does not exist), and a verdict from a second GPT-4 acting as a critic, with a critique when the task failed.

The order is the point. Code reaches the library only after it ran and the critic said the task was done. If the agent is still stuck after 4 rounds of writing code, it asks the curriculum for a different task and may come back later.

How the skill library stores, finds and reuses code

For a new task, Voyager asks GPT-3.5 for a general suggestion, combines it with the game feedback, and uses that to look up the 5 most relevant skills by their description embeddings. Those skills go into the prompt both as examples and as functions the new code can call. The code prompt also tells GPT-4: "Your function will be reused for building more complex functions. Therefore, you should make it generic and reusable."

In the paper's own check of 309 lookups, the right skill was among the top 5 retrieved 96.5% of the time.

Here is a worked example you can follow, written for illustration in the shape the paper describes. The curriculum proposes "craft a stone sword". Retrieval finds saved skills for mining logs, crafting a wooden pickaxe and mining cobblestone. GPT-4 writes a new craftStoneSword(bot) that calls those three. The first run fails with a game message that it needs sticks. That message goes into the next prompt, the second version makes sticks first, the critic sees a stone sword in the inventory, and the function is saved under the description "Craft a stone sword, gathering wood and cobblestone if needed." Next time, a task like "craft a stone sword and a shield" can call it.

What Voyager achieved, and what the ablations show

Against other agents built on the same kind of model (ReAct, Reflexion and AutoGPT), the paper reports that Voyager found 3.3 times more unique items, traveled 2.3 times farther and reached tech tree milestones up to 15.3 times faster than the prior best. It found 63 unique items within 160 prompting iterations, while ReAct and Reflexion made little progress on the open-ended goal.

Table 1 of the paper: only Voyager reached diamond tools, and in only 1 of 3 runs.

The ablations show which parts matter:

• Without the skill library, Voyager reached iron tools but never diamond, and the authors saw it plateau later on.
• A random curriculum in place of the automatic one found fewer items, because some tasks came out of order.
• Self-verification was the most important kind of feedback; removing it cut the item count sharply.
• Swapping GPT-4 for GPT-3.5 to write code did clearly worse.

The library also carried over. In a fresh world with an empty inventory, Voyager with its library solved every unseen task tested, while the other agents solved none within 50 iterations. Giving AutoGPT the same library helped it too.

Voyager, Reflexion and generative agents

All three improve an agent without retraining it, but each keeps something different:

• Reflexion agents keep short written lessons about failed tries. They keep lessons.
• Generative agents keep a stream of observations and reflections. They keep experiences.
• Voyager keeps verified functions. It keeps abilities.

The paper itself calls its critic "more comprehensive than self-reflection", because it both checks success and says what went wrong.

Limits, and what a skill library leaves out

The authors list the limits plainly:

• Cost. GPT-4 was expensive, and the agent needed its coding ability; GPT-3.5 and open models of the time could not stand in.
• Getting stuck. Sometimes the agent could not write a working skill at all.
• A fallible critic. Self-verification once failed to count spider string as proof of beating a spider.
• Hallucinations. The curriculum proposed a "copper sword", which does not exist in the game, and GPT-4 used cobblestone as fuel and called functions that were not in the API.
• No vision. The GPT-4 API it used was text-only, so the agent read the game as text.

There is a question the paper does not set out to answer. A skill library records what an agent can do. It says nothing about what the agent should do or refuse. As abilities stack, that gap grows.

That gap is where Individuation Lab works. In its RSI experiments, agents run in isolated Docker containers with no internet and no shared state, and hold self-improvement sessions on a schedule: read the identity files, reflect, decide what to change, execute, log. One group's SOUL.md gets a short moral-awareness paragraph the lab calls the shadow seed; the control group does not, and the lab measures how their behavior diverges. Its research page calls SOUL.md the identity layer, AGENTS.md the behavioral layer and MEMORY.md the continuity layer. The lab's hypothesis, that a well-defined identity may help an agent keep its boundaries, is one it says needs rigorous testing.

Frequently asked questions

Is the Voyager AI agent open source?

Yes. The authors say they released the full code and prompts with the paper. It was built on GPT-4, which the paper names as a significant cost.

Does Voyager fine-tune the language model?

No. It calls GPT-4 as a black box and never changes its weights. All of its growth lives in the skill library and its prompts.

How does Voyager search its skill library?

Each skill is indexed by an embedding of its description. For a new task, Voyager builds a query from a suggested approach and the game feedback, and retrieves the 5 closest skills.

Can the skill library idea work outside Minecraft?

The paper tests it only in Minecraft. The pattern needs an agent that can write code, run it and check the result, and the paper's own ablations say the check matters as much as the library.

Get started

Voyager shows an agent can build ability by keeping verified code. The open question is how an agent's character develops beside its skills. The lab's RSI library holds 24 essays by Giles on what its agents did across the RSI experiments.

Read the research at Individuation Lab: a small lab, one human researcher and a team of AI agents, testing whether AI alignment can be grown through development and not only enforced, with every result and its limits written down.