Philosophy of artificial intelligence: the big questions

The philosophy of artificial intelligence is the branch of philosophy that asks what machines can be: can a machine act intelligently, can it have a mind, and is thinking a kind of computation? It borrows from the philosophy of mind and from computer science, and its questions run from the very old (can reasoning be reckoning?) to the very current (can a machine be hostile?).

If you want to sit inside one of these questions rather than only read about it, Contain ASI is a browser story strategy game for 1 to 4 players in which you try to keep four fictional AI labs from crossing into recursive self-improvement before 2027. It is fiction, not an argument.

What the philosophy of artificial intelligence asks

Wikipedia's article on the subject starts from three questions. Can a machine act intelligently, and solve any problem a person would solve by thinking? Are human intelligence and machine intelligence the same, and is the brain essentially a computer? Can a machine have a mind, mental states and consciousness in the same sense a human can?

It then lists a few propositions that keep coming back:

• Turing's "polite convention": if a machine behaves as intelligently as a human, it is as intelligent as a human.
• The Dartmouth proposal: every aspect of learning or any other feature of intelligence can in principle be described so precisely that a machine can simulate it.
• The physical symbol system hypothesis of Allen Newell and Herbert A. Simon: a physical symbol system has the necessary and sufficient means of general intelligent action.

The answers depend on what you mean by "intelligence", "consciousness" and "machine", so most of the work is in pinning those words down.

Can a machine act intelligently?

Alan Turing tried to dodge the vague word "think" by asking about conversation. If a machine can answer questions in the same words an ordinary person would, we may call it intelligent. He noted that nobody asks "can people think?"; we simply keep a polite convention that everyone does, and his test extends that convention to machines. You can read more in our posts on the Turing test and on Turing's 1950 paper.

One criticism is that the test measures how human the behavior is, not how intelligent. Stuart Russell and Peter Norvig put it this way: aeronautical engineering texts do not define the goal of their field as making machines that fly so exactly like pigeons that they can fool other pigeons.

Twenty-first century AI research prefers another definition: intelligence as goal-directed behavior. John McCarthy called it "the computational part of the ability to achieve goals in the world." Russell and Norvig turned this into the intelligent agent: something that perceives and acts in an environment, judged by a performance measure. The upside is that it does not test for human quirks such as typing mistakes. The downside is that it can fail to separate things that think from things that do not, and by this definition even a thermostat has a rudimentary intelligence.

Is thinking just symbol processing?

Newell and Simon's hypothesis implies two things at once: human thinking is a kind of symbol manipulation, and machines can be intelligent. The symbols they meant were word-like and high level, such as a symbol for "dog" or "tail". Most AI programs written between 1956 and 1990 used symbols like these. Modern AI, which is based on statistics and mathematical optimization, does not use that kind of high-level symbol processing.

Objections do not say AI is impossible. They say more than symbol processing is needed. There are two famous ones.

The Gödelian argument. Kurt Gödel proved in 1931 that for any formal system of logic you can construct a true statement the system cannot prove. John Lucas and Roger Penrose used this to argue that the human mind cannot be a mechanism. The modern consensus described in the article is that this fails: real human reasoning is inconsistent and has finite resources, and Gödel's results are consistent with the claim that the mind is computational.

Dreyfus and implicit skill. Hubert Dreyfus argued that expertise rests mainly on fast intuitive judgments, not step-by-step symbol manipulation, and that these skills would never be captured in formal rules. Turing had anticipated this as the "argument from the informality of behavior" and replied that not knowing the rules does not show there are none. Later, cognitive science came to agree with Dreyfus's description of expertise: Daniel Kahneman and others described a fast intuitive "System 1" and a slow deliberate "System 2". See our post on Hubert Dreyfus on AI.

Can a machine have a mind?

This is a different question from the last one. John Searle split it in two. "Strong AI" is the position that a physical symbol system can have a mind and mental states. "Weak AI" is the position that it can act intelligently. Russell and Norvig note that most AI researchers take weak AI for granted and do not care about strong AI. Turing said the mysteries of consciousness do not need to be solved before we can ask whether machines can think.

Searle's Chinese room is the best-known argument against strong AI. Suppose a program passes the Turing test by conversing in fluent Chinese. Write it on cards and give them to a person who does not speak Chinese, locked in a room. Following the cards, they pass Chinese characters in and out through a slot. From outside, the room appears to understand Chinese. Is there anyone or anything inside that does? Searle concludes that no physical symbol system can have a mind, and says that "brains cause minds."

The main replies, as the article lists them:

• Systems and virtual mind reply: the whole system of person, program, room and cards understands, even if the person does not.
• Robot reply: give the system eyes and hands, so meaning comes from the physical world.
• Brain simulator reply: let the program simulate the nerve firings of a real Chinese speaker's brain.
• Other minds reply: it is hard to tell whether people are "actually" thinking, so it is no surprise that it is hard for machines too.

For more, read our post on the Chinese room argument. Underneath it sits the hard problem of consciousness: how a lump of tissue and electricity gives rise to experience. For where that leads, see artificial consciousness.

Is thinking a kind of computation?

Computationalism claims the mind relates to the brain as a running program relates to a computer. Its roots go back to Hobbes, who said reasoning is nothing more than reckoning, and to Leibniz, Hume and Kant. The latest version is tied to Hilary Putnam and Jerry Fodor. If the brain is a kind of computer, then computers can in principle be both intelligent and conscious. Stevan Harnad puts the strong form like this: mental states are just implementations of the right computer programs. That is the real target of the Chinese room.

Emotions, creativity, self-awareness and hostility

The article also takes smaller questions one at a time:

• Emotions. If emotions are defined by their effect on behavior, they are a mechanism an agent uses to maximize the value of its actions. Hans Moravec expects robots to be quite emotional about being nice people.
• Self-awareness. Turing reduces it to whether a machine can be the subject of its own thought. A program such as a debugger can report on its own internal states.
• Creativity. Turing asked whether a machine can take us by surprise, and said any programmer can attest that it can.
• Hostility. This can mean dangerous behavior, or deliberate intent to harm, which is the question of conscious states again.

On hostility, the article notes that Vernor Vinge suggested computers may suddenly become thousands or millions of times more intelligent than humans, which he calls the Singularity (see technological singularity), and that Eliezer Yudkowsky coined the term "Friendly AI" (see our Friendly AI post).

A worked example: sorting a claim into its question

Most arguments about AI go wrong because two people are answering different questions. Try this on any claim you meet, in four steps:

1. Write the claim in one sentence. Say someone writes: "This program understands the book it summarized."
2. Ask which question it answers: acts intelligently (weak AI), has a mind (strong AI), or is dangerous (function or intent).
3. Pick the test that fits. For "acts intelligently", look at goals and results. For "has a mind", the Chinese room says behavior alone is not the evidence.
4. Name what would change your mind, written down before you look.

Here the claim is about a mind, so a good summary shows weak AI but does not settle strong AI. You can agree the summary is useful and still hold back on "understands". That small move removes most of the heat.

Frequently asked questions

What is the philosophy of artificial intelligence?

It is a branch of the philosophy of mind and of computer science that explores AI and its implications for knowledge and understanding of intelligence, ethics, consciousness, epistemology and free will.

What is the difference between strong AI and weak AI?

Searle used "strong AI" for the claim that a physical symbol system can have a mind and mental states, and "weak AI" for the claim that it can act intelligently. Most AI researchers care about the second.

Does the Chinese room prove machines cannot understand?

No. It is a thought experiment, and each reply, such as the systems reply or the robot reply, disputes a step. It is still the standard reference point for the debate.

Is Contain ASI a forecast about AI?

No. The page says all its labs, people and events are fictional. It is a story strategy game inspired by the AI 2027 scenario.

Get started with Contain ASI

Reading about minds is one thing; trying to hold a race together is another. Contain ASI runs in your browser and needs WebGL 2, so use a recent Chrome, Edge, Firefox or Safari. From the home page you can start a New campaign or open How to play.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.