Build Measure Learn Loop for an AI Product, With an Example

The build measure learn loop is a simple cycle for a young product: build the smallest version that tests your idea, measure what real people do with it, and learn what to change before you build more. For an AI product the loop works the same way, with a few extra numbers worth watching, because a model can look impressive in a demo and still disappoint in daily use.

The first lesson of the Launch, Price and Get Paid course on Age of AI Entrepreneur is called "The MVP and the Build-Measure-Learn Loop", and a reader signs in with an email and an emailed code to open it. This post shows the loop with an AI example you can copy.

Where the build measure learn loop comes from

The loop belongs to the lean startup method, first proposed in 2008 by Eric Ries. In that method, a minimum viable product, or MVP, is the version of a new product that lets a team collect the most validated learning about customers with the least effort. The loop stresses speed: build an MVP of the idea, measure its effect in the market, and learn from the experiment.

Speed matters because your first guess about what people want is probably partly wrong. The faster you go round the loop, the sooner you find out which part.

Build: the smallest AI version that tests one belief

Start by writing the belief you are testing, in one sentence. For example: "Freelancers will send an AI-written follow-up email after a client call if the draft needs only a quick edit."

Then build only what tests that belief. For an AI product that is often one prompt, one page and a handful of real users. You do not need accounts, billing, a dashboard or a custom model. If you want a guide to that scale of build, our post on building your first AI product in a weekend sets out a plan.

Measure: three numbers an AI product needs

Counting sign-ups is not enough. Add numbers that show whether the model's output is useful and affordable.

• Use: how many people came back and ran it a second time. Repeat use tells you more than a first try.
• Quality: how often the output was used as it was, edited a little, or thrown away. Ask users to mark each result, or log what they change.
• Cost: what each run cost you in model fees. Our guide to unit economics for an AI product shows how to turn that into a figure per customer.

Write down what result would count as a pass before you look at the data. Otherwise any number can be explained as good news.

Learn: what the numbers tell you to change

Each pattern points at a different next move.

• Few people try it: the promise or the page is unclear. Change the message, not the model.
• People try it once and leave: the output was not good enough, or the job was not painful enough. Read the results they discarded.
• People use it but edit heavily: the model is close. Improve the prompt, add examples of what good looks like, or narrow the task.
• People love it but the cost per run is high: the idea works, so work on price or on a cheaper model, and test again.

Pick one change per round. If you change five things, you will not know which one moved the numbers.

A worked example: two rounds for a follow-up email tool

You build a page where a freelancer pastes call notes and gets a draft follow-up email. The numbers below are invented to show the method.

Round 1. You give it to 10 freelancers for a week. Seven run it twice or more, so use passes your bar. But only 3 in 10 drafts go out with a light edit, and the edits are almost all the same: the tone is too formal. So the lesson is not "the idea is wrong". It is "the draft sounds stiff".

Round 2. You change one thing: the prompt now includes two short examples of a friendly email, written by the users themselves. Same 10 people, another week. Now 7 in 10 drafts go out with a light edit. Cost per run is unchanged because the examples add only a few words. You then have a reason to take the next step, which might be a price test or a bigger group.

If quality had stayed low after the examples, that would also be learning: the task may be too open for a one-prompt tool. Our post on how to evaluate whether your AI feature actually works goes further on judging outputs, and the Evals course at its course page has a lesson on building an eval set.

Where builders disagree about the loop

Not everyone agrees that tiny first versions are the right move. Some argue that a rough first version can put people off for good, so a founder in a crowded field should ship something polished. Others say that polish is a bet placed before you know what people want. The course lists sources in every lesson, and the app sets out 17 cards on questions where researchers disagree, so you can read the arguments yourself.

Frequently asked questions

What is the build measure learn loop?

It is a cycle from the lean startup method: build a minimum version, measure how people use it, then learn what to change. You repeat it, going faster each time.

How long should one loop take?

As short as you can make it while still getting real use, often one to two weeks for a small AI tool. A shorter loop gives you more rounds to learn from.

What should I measure for an AI product?

Measure repeat use, how often the output is used as it is, and what each run costs. Together they show whether people want it, whether the model is good enough, and whether you can afford it.

Is an MVP the same as a prototype?

Not quite. A prototype may only show how something could work, while an MVP is used by real people so you can learn from what they do.

Get started

Open the Launch, Price and Get Paid course for its three short lessons: The MVP and the Build-Measure-Learn Loop; Pricing AI Products; and Launching and the First Ten Customers. You sign in with your email and an emailed code.

Start Age of AI Entrepreneur: short hands-on lessons on how to build a business in the age of AI, with every side of the debate and the sources to read next.