Wizard of Oz MVP: Test an AI Product Before You Build It

A Wizard of Oz MVP is a test where your product looks automated to the user, but a person does some or all of the work behind it. For an AI product, that person is usually you, with a chat assistant open in another tab. A wizard of oz mvp lets you find out whether people want the result before you spend weeks wiring up models, tools and code.

Age of AI Entrepreneur teaches this path from idea to first users in short hands-on lessons, such as the Your First Builds course, which you open after signing in with your email and an emailed code. Its blog post on how to build your first AI product without writing code covers picking a problem and no-code tools; this post covers the step before the tools, testing the idea by hand.

What is a Wizard of Oz MVP?

The name comes from research on how people use computers. In a Wizard of Oz experiment, people use a system they believe works on its own, while a hidden person runs part of it. The phrase was coined by John F. Kelley around 1980 during his dissertation work at Johns Hopkins University, after the man behind the curtain in L. Frank Baum's 1900 novel The Wonderful Wizard of Oz.

Founders borrowed the idea. An MVP, a minimum viable product, is the smallest thing that tests whether your idea is worth building. A Wizard of Oz MVP is an MVP where the clever part is faked by a person for now. The user fills in a form, sends an email or uploads a file, and gets a result back. They judge the result, not how it was made.

A close cousin is often called a concierge test. There, the user knows from the start that a person is doing the work for them. The two differ in what the user is told, which matters more than it seems, as the section on honesty below explains.

Why it suits a first AI product

An AI product has two risks at once. Will people want the result? And can a model produce that result reliably? Building first answers the second question and leaves the first one open. A Wizard of Oz MVP answers the first question first, and gives you real examples for the second.

• It costs days, not months. A form, an inbox and your own time are enough.
• You see every request. Each one shows you what people actually ask for, in their words.
• You collect test cases. Every request and the answer you sent becomes an example you can test a real model against later.
• You learn where a person is needed. Some steps stay with a human even after you automate the rest.

A worked example: one week, one tool, run by hand

Say your idea is a tool that turns a freelancer's messy call notes into a clean project proposal. Here is a week you can follow.

1. Day 1: the front door. Make a one-page site with a short promise ("Paste your call notes, get a proposal draft by email within a day") and a form with three fields: name, email, notes. The first lesson of the Your First Builds course, Project 1: A Landing Page and a Waitlist, walks through a page like this.
2. Day 1: the honest line. Under the form, write: "A person reviews every draft during our early test." That keeps you honest and takes the pressure off.
3. Days 2 to 5: be the wizard. Share the page with 20 freelancers you can reach. When notes come in, paste them into a chat assistant with a prompt you write and refine as you go, fix the draft by hand, and send it back.
4. Every request: write it down. Keep a sheet with the notes, the prompt you used, what you had to fix, and how long it took.
5. Day 6: ask. Email each person who used it: did they send the proposal? What did they change? Would they use it again next week?
6. Day 7: decide. Look at the sheet and make one call: stop, change the promise, or start automating the step you fixed least often.

Notice what you did not build: no login, no database, no model calls from code. If no one fills in the form, you have learned that for the price of one page.

What to measure in a Wizard of Oz MVP

Count a few things, the same way every time, so the decision on day 7 is not a feeling.

• Requests: how many people sent something, out of how many saw the page.
• Repeat use: how many came back with a second request without being asked.
• Your fixes: for each draft, did the chat assistant's first version need no change, small changes, or a rewrite? This tells you how far a model alone gets.
• Your minutes: time per request. If it is 40 minutes, the product will need a lot of automation to work as a business.
• What they did with it: a draft that was actually sent to a client is worth more than a polite "looks great".

The fixes column is the bridge to building. When you do automate, those examples become your first test set. Our post on the build measure learn loop for an AI product shows how to turn one round of results into the next test.

Where it goes wrong, and how to stay honest

The method has real limits. It does not scale: past a few dozen requests a week, you are running a service, not a test. It can hide the hard part: a person fixes things a model would get wrong, so a good result by hand does not prove a model can match it. And it can teach the wrong lesson if you only test with friends who are kind to you. The lesson Talking to Customers Without Fooling Yourself, in the Finding AI Opportunities course, and our list of customer discovery questions help with that last one.

Then there is honesty. In research, participants are sometimes not told a person is involved, and sometimes they are. With real customers, especially paying ones, hiding a person behind an "AI" label can break trust when it comes out, and people may share private details they would not give a stranger. The simple fix is the one in the worked example: say that a person reviews the results during the test. Most people do not mind, and some like it.

Frequently asked questions

Is a Wizard of Oz MVP the same as a concierge MVP?

They are close. In a concierge test the user knows a person does the work; in a Wizard of Oz test the product looks automated. For an AI product, saying a person reviews results keeps you honest either way.

How many users do I need for a Wizard of Oz MVP?

Enough to see a pattern, and few enough to handle by hand. Ten to twenty real requests in a week is a workable start for one person.

When should I stop doing it by hand?

When people come back for more and your notes show which step you fix least. Automate that step first, then the next.

Do I need to know how to code to run one?

No. A page builder, a form, an inbox and a chat assistant are enough. Code comes later, once you know what to build.

Get started

Open Age of AI Entrepreneur, sign in with your email and an emailed code, and work through the Your First Builds course, starting with a landing page and a waitlist, then the Finding AI Opportunities course for customer conversations. The terms page says every lesson is free to anyone with an account.