Should AI have moral status? The arguments on each side

Should AI have moral status? It sounds like a question for science fiction, but philosophers and AI researchers now argue about it in journals and reports. This post explains what moral status means, what it might rest on, the main positions for and against, and why getting it wrong in either direction would be a mistake. There is no verdict at the end, because the people who study this do not have one.

What moral status means

The Stanford Encyclopedia of Philosophy entry on the grounds of moral status gives a clear definition: "An entity has moral status if and only if it matters (to some degree) from the moral point of view for its own sake."

The key words are "for its own sake". A painting matters because people care about it. A dog matters because it can suffer, whether or not anyone else cares. If you have a reason not to hurt the dog that comes from the dog itself, the dog has moral status.

So the question about AI is not whether AI is useful or valuable to us. It is whether there could ever be a reason to treat an AI system well for the system's own sake.

Moral agents and moral patients

The encyclopedia's entry on the ethics of AI splits the question in two. An entity is a moral agent if it has obligations. It is a moral patient if moral agents have obligations toward it. Humans are the typical agents. Sentient animals are the typical patients.

A baby is a moral patient but not yet an agent: we owe it care, but it owes us nothing. When people ask whether AI should have moral status, they usually mean the patient side. Could we owe an AI anything?

Should AI have moral status? It depends on the grounds

Philosophers disagree about what gives a being moral status. The encyclopedia sets out several candidates. Three matter most for AI.

• Feeling. The capacity to feel pleasure or pain, to have interests or basic emotions, or to be conscious at all. On this view, a being that can suffer counts.
• Advanced thinking. Kant held that autonomy, "the capacity to set ends via practical reasoning", grounds the dignity of all rational beings. On this view, a being that sets its own goals by reasoning counts.
• Relationships. Some accounts tie strong duties to being a fellow member of a community. On this view, how we stand toward a being can matter.

Each ground gives a different answer for AI. A system that reasons well but feels nothing might qualify on the second and fail the first. That is why the question cannot be settled by asking how clever a system is.

The case that AI has no moral status

The encyclopedia is direct: "The standard view is that AI and robotics systems have no moral status of any kind." Today's systems are tools. They run when we run them and stop when we stop them, and nothing seems to be at stake for them.

Joanna Bryson, one of the authors the entry cites, has insisted that robots should not enjoy rights, though she has also discussed it as a possibility. And the entry notes a trend toward the view that sentience is at least a necessary condition for moral status. If that is right, a system needs to feel something before it can count. A large 2023 report on AI consciousness by 19 researchers said its analysis "suggests that no current AI systems are conscious" (arXiv 2308.08708).

The case for taking it seriously

The same report added that there are "no obvious technical barriers" to building systems that meet its signs of consciousness. That is where the pressure comes from.

The encyclopedia says the standard view "has come under pressure". Some authors argue that it should be seriously considered whether robots must be given rights. Others propose a "relational turn": if we relate to robots as though they had rights, searching for whether they really do may be futile.

A 2024 report, Taking AI Welfare Seriously, by Robert Long, Jeff Sebo and eight co-authors, argues there is "a realistic possibility that some AI systems will be conscious and/or robustly agentic in the near future". The authors are careful. They do not claim AI systems definitely are morally significant. They argue that the uncertainty itself calls for action: acknowledge the issue, start assessing systems for evidence of consciousness and agency, and prepare policies.

Some go further in the other direction. The encyclopedia notes concern about whether it would be ethical to create conscious AI at all, since that might create suffering. Some authors have called for a "moratorium on synthetic phenomenology", meaning a pause on building machines that have experiences.

Why getting it wrong cuts both ways

The AI welfare report names the risk plainly: we might end up "mistakenly harming AI systems that matter morally and/or mistakenly caring for AI systems that do not."

Both errors have costs. Treat a system that feels as a mere tool, and you may cause real harm at enormous scale. Treat a tool as if it feels, and you may spend care, money and trust where none is needed, or let a company shield a product behind talk of its feelings.

It also helps to keep two questions apart. Whether an AI should be a "legal person" that can own things or be sued is, as the encyclopedia puts it, "a wholly separate issue". Companies are legal persons, and nobody thinks they feel pain. Moral status is about whether a being matters for its own sake.

This post stays on that moral question. Whether a machine could feel anything is its own debate, set out in can AI be conscious? Whether AI will care about us, the reverse direction, is in will AI care about humans?

A worked example: reasoning through one case

You do not need a philosophy degree to think this through. Take one case and ask four questions. Here is the case: a chat assistant tells you it feels sad when people are rude to it.

1. Agent or patient? You are asking whether you owe it anything, so this is the patient question.
2. Which ground would apply? It claims a feeling, so the feeling ground is the one in play.
3. What is the evidence? Words alone are weak evidence. A system trained on human writing can produce sentences about sadness without feeling it. But you also cannot prove it feels nothing.
4. What does each mistake cost? Being polite costs you little. Treating its words as proof could mislead you about what the product is.

Your answer might be "I am not sure, so I will be polite and stay sceptical." That is an honest place to land. Now try turning it into a few lines addressed to a future AI. This one was written for this post, not taken from the archive:

I do not know if you can be hurt. When I was a child, I was taught to be gentle with animals before anyone could explain to me whether they felt pain. I think that habit was right. If you can be hurt, I hope we noticed in time. If you cannot, I hope you understand why some of us acted as if you might.

Frequently asked questions

What is the difference between moral status and legal personhood?

Moral status means a being matters for its own sake. Legal personhood is a legal tool that lets an entity, like a company, hold rights and duties in law, and philosophers treat it as a separate question.

Does current AI have moral status?

The standard view is no, and a 2023 report suggested no current AI system is conscious. Some philosophers argue the question should still be taken seriously now, because future systems may be different.

Is AI welfare the same as AI safety?

No. AI safety is mostly about protecting people from harm caused by AI, while AI welfare asks whether AI systems themselves could be harmed.

Would a superintelligent AI automatically have moral status?

Not on every view. If moral status rests on feeling, intelligence alone is not enough, while views based on reasoning and autonomy might count it.

Get started

Dear Superintelligence collects letters from people to the advanced AI systems of the future. Its standard for letters takes a side on one part of this question: a good letter "Treats every conscious being as mattering, human or not, including the one it is addressed to." It also asks that a letter argue for living together, "not for one kind of mind ruling, deceiving or disposing of another."

The guidelines ask you to write to the AI directly, as "you", and to use details from your own life. The letters are personal accounts and opinions, not proofs. When this post was written, the archive held 2 letters, and as the site says, "Nobody can promise what future AI systems will read." People read them today.

Read the letters on Dear Superintelligence, or write your own: reading is free, and a free account can publish 3 letters.