Herbert A. Simon: what he contributed to AI

Herbert A. Simon was an American scholar who helped start artificial intelligence: with Allen Newell he built two of the first problem-solving programs, the Logic Theory Machine in 1956 and the General Problem Solver in 1957. He is best known, though, for two ideas about decisions, bounded rationality and satisficing. This post explains what he built, what those ideas mean, what he predicted wrongly, and why his questions about limited minds still matter.

If you would like to think about decisions made with too little time and information inside a story, Contain ASI is a browser story strategy game for 1 to 4 players where you try to keep four fictional AI labs from crossing into recursive self-improvement before 2027.

Who Herbert A. Simon was

Wikipedia's article on Herbert A. Simon describes him as an American scholar whose work influenced computer science, economics and cognitive psychology. He was born on June 15, 1916, in Milwaukee, Wisconsin, and died on February 9, 2001, in Pittsburgh, aged 84. He took all three of his degrees at the University of Chicago.

From 1949 until his death he taught at Carnegie Mellon University in Pittsburgh. His doctoral students included Allen Newell, his long-time partner in AI, and Edward Feigenbaum. The article lists his fields as economics, artificial intelligence, computer science and political science. His main research interest, it says, was how decisions are made inside organizations.

He received two of the highest honors in two different fields: the ACM Turing Award in 1975, shared with Newell, and the Nobel Memorial Prize in Economic Sciences in 1978.

Bounded rationality: the idea that led him to computers

Simon's first big book, Administrative Behavior, came out in 1947 and grew from his doctoral dissertation. It asked how people in organizations actually make decisions. He put the problem sharply: if there were no limits to human rationality, the theory of administration would be barren. It would hold one rule only: always pick the alternative that best achieves your goals.

The ideal version of a rational choice, as the book set it out, has three steps. List every alternative. Work out every consequence of each one. Compare them all. Simon's point was that no real person or organization can do this. You face uncertainty about the future, and getting information costs time and money now.

So real deciders have only bounded rationality, and they decide by satisficing: choosing an option that might not be the best, but is good enough. Bounded rationality is now a central theme in behavioral economics. Wikipedia says he was the first to rigorously examine how administrators decide without perfect and complete information, and that this is the work his Nobel Prize recognized.

The Logic Theory Machine and the General Problem Solver

Decisions led Simon to programs. With Allen Newell he created the Logic Theory Machine in 1956 and the General Problem Solver, or GPS, in 1957. Wikipedia calls him a pioneer of the field for this work.

The article picks out one thing about GPS: it may have been the first method that separated a problem-solving strategy from the information about a particular problem. In plain words, the way of searching for a solution was written once, and the facts of each puzzle were fed in separately. For a later family of rule-based programs, and why rules fell short, see expert systems.

Both programs were written in the Information Processing Language, or IPL, which Newell, Cliff Shaw and Simon developed in 1956. IPL did list processing, and the computer scientist Donald Knuth notes that the linked list was first called "NSS memory," after its three inventors. The Turing Award citation sums up the partnership: over twenty years, first with Shaw at the RAND Corporation and then with colleagues at Carnegie Mellon, Newell and Simon made basic contributions to artificial intelligence, the psychology of human cognition, and list processing.

Programs as theories of the mind

Simon used programs to test ideas about people. With Newell he built a theory of human problem solving based on production rules, simple if-this-then-that steps. Studying how people solve problems needed new kinds of measurement, so with Anders Ericsson he developed verbal protocol analysis, a technique built on what people say as they work.

With Edward Feigenbaum he developed EPAM, the Elementary Perceiver and Memorizer. Wikipedia calls it one of the first theories of learning to be run as a computer program. Later, with Fernand Gobet, the model grew into CHREST, used mostly to simulate parts of chess expertise.

He also had a view on how experts are made. He said becoming an expert in a subject takes about ten years of experience, and he and colleagues estimated that expertise comes from learning roughly 50,000 chunks of information. A chess expert, on this account, has learned about 50,000 patterns of chess positions. Learning from stored experience is also the idea behind case-based reasoning.

Emotions, and the predictions Herbert A. Simon got wrong

In the early 1960s the psychologist Ulric Neisser argued that machines could copy "cold" thinking, such as reasoning, planning and deciding, but never "hot" thinking: pain, pleasure, desire and other emotions. Simon answered in 1963 with a paper on emotional cognition, updated in 1967 and published in Psychological Review. The AI community largely ignored it for years, until later work on emotions by Sloman and Picard drew attention back to it and it became highly influential.

His forecasts fared worse. In 1957 he predicted that computers would beat humans at chess within ten years; it took about forty. In 1965 he wrote that "machines will be capable, within twenty years, of doing any work a man can do." That did not happen on his schedule.

The lesson is not that Simon was careless. A founder of the field, who knew its programs from the inside, still misjudged how long hard problems would take. That is worth remembering when you read any confident date about AI, in either direction. For other routes AI research took beside Simon's symbolic programs, see computational intelligence.

Try it: satisfice one real decision

You can feel the gap between Simon's two columns in ten minutes. Pick a decision you are putting off, like which laptop to buy or which course to take.

1. Try the ideal first. Write how many options exist and how many facts you would need about each to be sure you picked the best. Notice how fast the list outgrows your week.
2. Write your limits. How much time, money and attention can you spend on this choice? Those are your bounds.
3. Set "good enough." Write two or three things the choice must do. Anything that does them all passes.
4. Search and stop. Look at options until one passes, then take it.
5. Check later. A month on, write whether good enough was good enough. That note is what you learn from next time.

Simon's claim is that this is not a lazy shortcut. It is how any real decider, person or organization, has to work.

Frequently asked questions

What did Herbert A. Simon contribute to AI?

With Allen Newell he created the Logic Theory Machine in 1956 and the General Problem Solver in 1957, both written in the IPL language he developed with Newell and Cliff Shaw. He also used programs to model human thinking and learning.

What is bounded rationality?

It is Simon's idea that real deciders cannot weigh every option and consequence, because the future is uncertain and information costs something. So they satisfice: they choose an option that is good enough rather than the best.

Did Herbert Simon win a Nobel Prize?

Yes. He received the Nobel Memorial Prize in Economic Sciences in 1978, for his work on how administrators decide without perfect information. He also shared the 1975 Turing Award with Allen Newell.

What did Simon predict about AI?

In 1957 he predicted computers would beat humans at chess within ten years, which took about forty. In 1965 he predicted machines could do any work a person can do within twenty years.

Get started with Contain ASI

Simon spent his life on one question: how do real people decide when they cannot know everything? Contain ASI asks it as fiction: as a researcher, research manager, CEO or government, you have three years to keep four fictional AI labs from reaching recursive self-improvement before 2027. Its Chapter 1, Before ASI, starts in January 2024, and the game says it is inspired by the AI 2027 scenario by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean. It needs a browser with WebGL 2.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.