Perceptrons book by Minsky and Papert: what it showed

The Perceptrons book is a 1969 work of mathematics by Marvin Minsky and Seymour Papert that proved what a simple learning machine, the perceptron, can and cannot compute. It is also one of the most argued-about books in the history of AI, because many people believe it helped push neural network research aside for more than a decade. This post explains what the book actually proved, how the famous XOR story gets told wrong, and what the argument still teaches.

If you like seeing how confident claims about AI play out over time, Contain ASI is a browser story strategy game for 1 to 4 players in which you try to keep four fictional AI labs from crossing into recursive self-improvement before 2027.

What the Perceptrons book is

Wikipedia's article on Perceptrons (book)) gives its full title as Perceptrons: An Introduction to Computational Geometry. Minsky and Papert decided to write it when Papert arrived at MIT in 1963. They meant it as a theoretical account of what perceptrons cannot do, and it took them until 1969 to solve the mathematical problems that kept turning up as they wrote.

The book has thirteen chapters in three parts. Chapters 1 to 10 set out the authors' theory of perceptrons through proofs. Chapter 11 is about learning, chapter 12 about separating things with straight lines, and chapter 13 gives the authors' own thoughts on simple and multilayer perceptrons and pattern recognition.

There are three versions. The first edition came out in 1969, a printing with the authors' handwritten corrections followed in the early 1970s, and an expanded edition appeared in 1988. That last one adds a prologue and an epilogue answering the 1980s revival of neural networks, but no new scientific results. A 2017 reprint carries a foreword by Léon Bottou, written from the point of view of someone working in deep learning.

The perceptron before the book

The perceptron was a neural net built by the psychologist Frank Rosenblatt in 1958, one of the most famous machines of its time. In 1960 Rosenblatt and colleagues proved that it could learn, in a finite number of training rounds, any task its settings were able to represent. That result is called the perceptron convergence theorem.

The hopes went far beyond that proof. The article records newspaper reports and statements by Rosenblatt claiming that neural nets would soon see images, beat humans at chess and even reproduce. At the same time, symbolic AI was rising, and the two camps competed for money, people and scarce computing power.

The rivalry was also personal, in a friendly way. Minsky and Rosenblatt had known each other since their teens at the Bronx High School of Science. They argued loudly at conferences and stayed friends, and the book is dedicated to Rosenblatt. If you want the wider picture of the two research styles, see neats and scruffies and symbolic AI.

What the book proved: parity and connectedness

Minsky and Papert studied an abstract version of the perceptron. It had a "retina" of inputs, a single layer of small input functions, and one output. They also limited how many inputs each of those small functions could look at, so that each one saw only a small patch of the picture. They called this conjunctive localness.

Their two main test cases were parity and connectedness. Parity asks whether the number of lit inputs is odd or even. Connectedness asks whether a shape on the retina is all one piece. The book proved that a single-layer perceptron with only local views cannot compute parity, and that the size of view needed to judge connectedness grows as the picture grows.

The cover of the book shows a connectedness puzzle in odd colors, chosen to show that people find this hard too. One reviewer, Earl Hunt, made a similar point about XOR: people also struggle to learn it in concept learning experiments.

The XOR affair: what the book did and did not say

XOR, short for exclusive or, outputs 1 when exactly one of two inputs is 1. A single perceptron cannot compute it, because it can only split its inputs with one straight line, and the XOR answers sit on opposite corners. That much is true and easy to check.

The popular story goes further: that the book claimed bigger networks share the same limit, so the whole idea should be dropped. Wikipedia's article calls this story full of mistakes. It was widely known that networks of such units can compute any logical function. Warren McCulloch and Walter Pitts knew it, Rosenblatt's own book mentions it, and so does Perceptrons.

What the book does prove is narrower. In a three-layer network, some patterns cannot be computed unless at least one unit in the middle layer is wired to every single input. That went against a hope of the time: that you could build useful networks out of local units, each seeing only a small patch, which were much easier to build. The researcher Jordan Pollack argued that a small proof about parity was read by the field as a successful attempt to bury the whole idea.

Did the book cause the decline of neural networks?

The book was well received at first. In 1969 Michael A. Arbib wrote that it had been widely hailed as an exciting new chapter in the theory of pattern recognition, and Allen Newell opened his review in Science with "this is a great book."

Critics pushed back on the definitions. H. D. Block said the book studied "a severely limited class of machines" from a viewpoint alien to Rosenblatt's, and called the title seriously misleading. Bernard Widrow said the perceptrons were defined too narrowly and that the proofs were "pretty much irrelevant", coming a decade after Rosenblatt's work.

The case that it did

The book is often thought to have caused a decline in neural net research in the 1970s and early 1980s, while symbolic AI grew fast. In a 1971 report to funders, Minsky and Papert wrote that virtually nothing was known about networks with hidden layers and that they believed such a network could do little more than a simple perceptron. When neural networks came back in the 1980s, David Rumelhart and colleagues wrote in 1986 that the authors' "pessimism about learning in multilayer machines was misplaced."

The authors' own answer

In the 1988 edition, Minsky and Papert argued that perceptron research faded because of its own problems, not because of their book. No learning machine of the time handled credit assignment better than Rosenblatt's rule, and perceptrons could not represent the knowledge some problems need. They called multilayer nets a "sterile" extension and argued that intelligence is a society of many small, different methods rather than one uniform machine. Minsky later compared the book to the Necronomicon from H. P. Lovecraft's stories: known to many, read by few.

Try it: test a perceptron by hand

You can check the core of the XOR argument with a pencil in five minutes. A perceptron here has two inputs, x1 and x2, two weights, w1 and w2, and a threshold t. It outputs 1 when w1 times x1 plus w2 times x2 is greater than t, and 0 otherwise.

1. Build AND. Set w1 = 1, w2 = 1, t = 1.5. Only the input (1, 1) gives a sum of 2, which beats 1.5. The other three give 0 or 1. It works.
2. Build OR. Keep the weights and set t = 0.5. Any input with a 1 in it now beats the threshold. It works too.
3. Try XOR. You need (0, 0) to give 0, so 0 is not greater than t, which means t is 0 or more. You need (1, 0) and (0, 1) to give 1, so w1 and w2 are each greater than t. Add those two facts: w1 plus w2 is greater than 2t, which is at least t. But (1, 1) must give 0, so w1 plus w2 must not be greater than t. Both cannot hold. No weights work.
4. Add a layer. Feed both inputs into an OR unit and a NAND unit (NAND is AND with the answer flipped: weights of -1 and a threshold of -1.5). Then feed those two outputs into an AND unit. Check all four inputs and you get XOR.

Step 3 is the limit that is true of one unit. Step 4 is why that limit never applied to networks in general. The real argument in the book was about step 4 under extra rules, such as each unit seeing only part of the input.

What the argument still teaches

Wikipedia's article notes that the way this book steered research first one way and then another has been the subject of a sociological study of how science develops. Both sides made confident claims: early hopes that perceptrons would soon see and play chess, and later doubts that layered networks would ever be worth much. The history is told in more detail in our short history of AI and in the post on Marvin Minsky.

The lesson people draw differs. Some read it as a warning against hype; others as a warning against declaring an approach dead too early. Both readings fit the record, and the article itself takes no side on how much the book alone was to blame.

Frequently asked questions

Who wrote the Perceptrons book?

Marvin Minsky and Seymour Papert. It was first published in 1969, with an expanded edition in 1988.

Did Perceptrons say neural networks could never compute XOR?

No. It showed that a single perceptron cannot, and it mentions that networks of such units can compute any logical function. Its harder results were about networks whose units each see only part of the input.

Did the book cause the first AI winter?

It is often blamed for a decline in neural net research in the 1970s and early 1980s. The authors argued that the research faded because of its own problems, and people still debate how much the book mattered.

Who invented the perceptron?

The psychologist Frank Rosenblatt, who published his first perceptron model in 1957 and developed the perceptron as a neural net in 1958. The book is dedicated to him.

Get started with Contain ASI

The Perceptrons story shows how hard it is to judge, in the moment, which AI ideas will matter. Contain ASI lets you explore that kind of uncertainty as fiction: as a researcher, research manager, CEO or government, you try for three years to keep four fictional AI labs from reaching recursive self-improvement before 2027. Its Chapter 1, Before ASI, starts in January 2024, and the game says it is inspired by the AI 2027 scenario by Daniel Kokotajlo, Scott Alexander, Thomas Larsen, Eli Lifland and Romeo Dean. It needs a browser with WebGL 2.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.