Case-based reasoning in AI: how machines learn from past cases

Case-based reasoning is a way for an AI system to solve a new problem by finding a similar problem it solved before and adapting that old answer. It is how a mechanic works when they recall another car that made the same noise, or how a lawyer argues from precedent. This post explains the four steps machines use, how the idea differs from learning general rules, where it came from, and where it falls short.

If you would like to think about how machines and people reason under pressure inside a story, Contain ASI is a browser story strategy game for 1 to 4 players where you try to keep four fictional AI labs from crossing into recursive self-improvement before 2027.

What case-based reasoning means

Wikipedia's article on case-based reasoning defines it, broadly, as the process of solving new problems based on the solutions of similar past problems. People often shorten it to CBR.

The article gives three everyday pictures of it. An auto mechanic fixes an engine by recalling another car with similar symptoms. A lawyer argues for an outcome from legal precedents, and a judge who creates case law adds to that store. An engineer who copies working parts of nature treats nature as a library of solved problems.

So CBR is a kind of reasoning by analogy. Some have argued that it is not only a useful method for computers but how people solve most everyday problems, and a more radical view holds that all reasoning rests on past cases a person has lived through. The article links that view to prototype theory in cognitive science. You do not have to accept the strong version to see the point: much of what you know how to do, you know because you did something like it before.

The four steps: retrieve, reuse, revise, retain

For computer reasoning, CBR has been set out as a four-step process. A case here is a stored record: a problem, its solution, and usually notes on how the solution was reached.

The article walks through the steps with a cook named Fred who wants to make blueberry pancakes.

1. Retrieve. Given the new problem, find relevant cases in memory. Fred is a novice, and the closest thing he remembers is making plain pancakes. That procedure, with his reasons for each choice, is the retrieved case.
2. Reuse. Map the old solution onto the new problem, adapting it where needed. Fred adds blueberries to his plain pancake method.
3. Revise. Test the new solution in the real world, or a simulation, and fix it. Fred mixes the blueberries into the batter and it turns blue. He revises: add the blueberries after the batter is in the pan.
4. Retain. Store the result as a new case. Fred records his blueberry pancake method, so the next time he faces a pancake problem he has more to draw on.

In a computer system, the retrieve step usually relies on a similarity measure: some way of scoring how close a stored case is to the new one. The retain step is where the system gets better. Its memory grows with each problem it solves.

Case-based reasoning vs rule induction: lazy and eager

At first glance CBR looks like rule induction, a family of machine learning methods that learn general rules from examples. Both start from a set of examples, and both generalize. The difference the article draws is about when they generalize.

A rule induction method generalizes before it knows what problem it will face. Give it recipes for plain, Dutch apple and banana pancakes, and it has to work out general rules for all pancakes up front. Only later is it asked for blueberry pancakes. The hard part is guessing in advance which way to generalize. The article calls this eager generalization.

CBR waits. It already knows the target is blueberry pancakes, so it generalizes from its stored cases exactly as far as this problem needs. That is lazy generalization, and it is why CBR tends to suit rich, complex domains where a case could be generalized in many different ways.

If you have read about logic programming or expert systems, the contrast is familiar. Those start from rules someone writes down. CBR starts from experience someone has stored.

Shallow knowledge and deep reasoning

In problem solving, the article says, CBR is also called a symptomatic strategy. It runs on knowledge gathered from experience that links symptoms to causes. The article lists several names for that kind of knowledge: shallow, compiled, evidential, history-based and case-based. It is the strategy most tied to diagnosis by experts, who recognize a problem quickly because they have seen ones like it.

The alternative is the topographic strategy, or deep reasoning. Here you use a detailed model of how a system is built and how its parts relate, reasoning from first principles. This deep, causal or model-based knowledge is what you fall back on for a fault you have never seen.

The article is clear about the limit: a symptomatic strategy will not work on its own for truly new problems, or when you want a deeper understanding of what is going on. Researchers Hoc and Carlier noted that each approach can support the other. Case-based hunches can suggest causes, and a deeper model can test them. The same tension runs through commonsense reasoning in AI: experience is fast, but it runs out where the world is new.

Try it: run the four steps on your own problem

You can feel how CBR works, and where it slips, with one problem of your own. Take ten minutes and a sheet of paper.

1. Write the new problem in one line. For example: "My bike chain keeps slipping on the hill to work."
2. Retrieve. List two past problems that felt similar. Next to each, write the one or two features that made you think of it. Those features are your similarity measure.
3. Reuse. Pick the closest case and write how its fix would change for this problem. Note every change you had to make.
4. Revise. Try the adapted fix, or imagine trying it, and write what could go wrong. Fred's blue batter is the kind of surprise to look for.
5. Retain. Write the final case as three lines: the problem, what worked, and what you would do differently.

Now look at step 2 again. Were the features you matched on the ones that actually caused the problem, or just the ones you noticed first? That gap is where CBR systems succeed or fail.

Criticism and the history of case-based reasoning

The main criticism the article records is that CBR takes anecdotal evidence as its operating principle. Without statistically relevant data behind it, there is no guarantee that its implicit generalization is correct. The reply given is that any inductive reasoning, where data is too scarce for statistics, rests on anecdotal evidence too. More recent work has tried to place CBR inside a statistical framework, so a case-based prediction can come with a stated level of confidence.

The history is short to tell. CBR traces its roots to Roger Schank and his students at Yale University in the early 1980s. Schank's model of dynamic memory was the basis for the first systems: Janet Kolodner's CYRUS and Michael Lebowitz's IPP. Through the 1980s other schools grew up around legal reasoning and memory-based reasoning, a way of reasoning from examples on massively parallel machines. In 1995 an International Conference on Case-Based Reasoning was set up, along with workshops in several countries.

The earliest deployed system the article names is Lockheed's CLAVIER, which laid out composite parts to be baked in an industrial convection oven. The article also names the Compaq SMART system, and says CBR found a major use in the health sciences and in structural safety management. For where CBR sits among AI's other traditions, see neats and scruffies.

Frequently asked questions

What is case-based reasoning in simple terms?

It is solving a new problem by finding a similar one solved before and adapting its answer. A mechanic recalling a car with the same symptoms is using it.

What are the four steps of case-based reasoning?

Retrieve similar cases, reuse the best solution by adapting it, revise it after testing, and retain the result as a new case for next time.

How is case-based reasoning different from rule learning?

Rule induction generalizes from examples before it knows the problem, which is eager generalization. Case-based reasoning waits until it has the problem and generalizes only as far as that problem needs, which is lazy generalization.

Who started case-based reasoning?

It traces to Roger Schank and his students at Yale in the early 1980s. Janet Kolodner's CYRUS and Michael Lebowitz's IPP were the first systems built on Schank's model of dynamic memory.

What is the main weakness of case-based reasoning?

It relies on past cases, so it struggles with truly new problems, and critics say it rests on anecdotal evidence with no guarantee its generalizations are right.

Get started with Contain ASI

Case-based reasoning works well until the problem has no precedent. Contain ASI puts you in a story like that, as fiction: as a researcher, research manager, CEO or government, you try for three years to keep four fictional AI labs from reaching recursive self-improvement before 2027. Its Chapter 1, Before ASI, starts in January 2024, and the game says it is inspired by the AI 2027 scenario by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean. It needs a browser with WebGL 2.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.