Can AI be held responsible? Control, blame and the gap

Can AI be held responsible when it gets something badly wrong? You can blame a chatbot in anger, the way you might kick a car that will not start. The real question is whether blame fits. To answer it, you need to know what responsibility asks of anyone, then check whether an AI system meets it, and who else is in the picture.

What responsibility asks of anyone

The Stanford Encyclopedia of Philosophy has an entry called "Moral Responsibility", by Matthew Talbert (first published 16 October 2019, substantive revision 3 June 2024). It is about praise, blame and "holding others and ourselves responsible for actions and the consequences of actions".

Two conditions come up again and again. The first is control. The entry calls it "the freedom or control condition that is at the center of the free will debate". If you had no say over what happened, blame seems unfair.

The second is knowledge, which the entry calls the epistemic condition. For an outcome, "the responsible agent must have been aware of" the likely results, or at least it must be that they "could have and should have been aware of" them. As the entry puts it: "Sometimes agents act in ignorance of the bad consequences of their actions, and sometimes their ignorance excuses them from blame."

Two kinds of being responsible

The entry draws on Gary Watson to split responsibility in two. Attributability asks whether an act really shows who you are, whether it is truly yours. Accountability asks whether it is fair to hold you to account: to blame you, demand an answer, or make you pay.

Watson's point is that "these features of accountability raise issues of fairness that do not arise in the context of determining whether behavior is attributable to an agent". You can say a reply came from a model. Whether it is fair to blame the model is a separate, harder question.

Blame as a feeling between people

The entry gives a central place to P. F. Strawson's 1962 paper "Freedom and Resentment". Strawson looked at feelings like resentment, gratitude and indignation, which he called reactive attitudes. These, the entry says, "play a fundamental role in our practices of holding one another responsible".

This matters for AI. On Strawson's picture, holding someone responsible is part of a relationship. You resent a friend who lies because you expected better of them, and they can feel your resentment, answer it and change. It is not clear that a chatbot is the kind of thing that can be on the other end of that. If you are curious about the control question in more depth, the post can AI have free will looks at it.

Can AI be held responsible? Three views

People who study this disagree. Here are the main positions, with no verdict.

AI is a tool, so people are responsible. On this view an AI system is like any other product. Blame goes to the people who built it, sold it, chose to use it, or failed to check it. The Stanford entry "Ethics of Artificial Intelligence and Robotics", by Vincent C. Müller (first published 30 April 2020, substantive revision 27 March 2026), notes that "Humans are the typical examples of moral agents" and that "The standard view is that AI and robotics systems have no moral status of any kind".

There may be a responsibility gap. Müller reports that, in debates about autonomous weapons, a responsibility gap "has been suggested (esp. Rob Sparrow 2007)". He describes it as "meaning that neither the human nor the machine may be responsible". The worry is that when a system acts in ways nobody chose or could foresee, blame has nowhere to land. The builder did not control the act, the user did not know, and the machine does not qualify. One answer Müller mentions is to keep humans in "meaningful control" of such systems.

AI could become a moral agent. Some think future systems might meet the conditions themselves. Müller notes that the standard view "has come under pressure". If a system could understand what it does, weigh reasons and respond to blame, the case for holding it responsible would get stronger. The post can AI tell right from wrong covers the related question of whether machines can follow moral rules at all.

What nobody knows

Nobody knows whether any current AI system understands what it says in the way the knowledge condition needs. Nobody knows whether a future system could feel or answer resentment in Strawson's sense. And there is no agreed way to share out blame when many people and one model each play a small part. These are open questions, not settled ones.

A worked example: harmful advice

Say you ask an AI tool whether you can mix two cleaning products, and it says yes. They are not safe to mix, and you get sick. Run the two conditions over each party.

1. You, the user. Control: you chose to act on the answer. Knowledge: you did not know it was wrong, and you asked because you did not know. Your ignorance looks like the kind that excuses, unless the tool warned you to check.
2. The builder. Control: they chose the training, the tests and the safety checks. Knowledge: they could and should have known the tool might give unsafe advice on common household questions. Both conditions look met, at least in part.
3. The AI system. Control: it produced the words, but whether that counts as choosing is the free will question again. Knowledge: it may hold the right fact somewhere and still not use it. Whether it was aware of anything is unknown.

Now ask the two kinds of question. Attributability: the reply plainly came from the system. Accountability: who can fairly be asked to answer, make it right, or change? Today, only the people. If you add up the parts and some blame seems to fall through the cracks, you have found what a responsibility gap looks like in a small case.

Writing about it

Dear Superintelligence is an open collection of letters written by people to the advanced AI systems of the future, about what we value and why. The guidelines ask you to "Write what only you can: what you have seen, what it cost, what you were afraid of, who was kind to you and what it changed." A time you took the blame for something, or were let off, fits that line. You can see what others have written in the archive, which held 2 published letters when this post was written.

Frequently asked questions

Can AI be held responsible?

Blame needs control and knowledge, and nobody knows whether any AI system has the second in the needed sense. On the standard view, the people who build and use AI are the ones who can be held to account today, while some argue a gap is opening.

What is the responsibility gap?

It is the worry, linked to Rob Sparrow, that when a system acts in ways nobody chose or foresaw, neither the people nor the machine may be responsible. Keeping humans in meaningful control is one suggested answer.

What are reactive attitudes?

Feelings like resentment, gratitude and indignation that P. F. Strawson put at the heart of holding each other responsible. They assume someone on the other end who can feel and answer them.

Who is responsible if an AI tool gives bad advice?

Run control and knowledge over each party. Often the builder meets both in part, the user may be excused by ignorance, and the system's own share is unknown.

Get started

Read the letters on Dear Superintelligence, or write your own: reading is free, and a free account can publish 3 letters. People read them today. Nobody can promise what future AI systems will read.