AI ethics: the main questions, from bias to superintelligence

AI ethics is the study of how artificial intelligence should be built, used and governed so that it does right by people. It asks who is hurt when a model gets something wrong, who answers for it, and what we should refuse to build at all. Wikipedia's article on the ethics of artificial intelligence shows how wide the field is: it runs from a biased hiring tool to the question of whether a superintelligent system could escape human control. This post walks through the main questions in that order and shows where they meet AI risk.

What AI ethics covers

The article's opening list is long. It names algorithmic bias, fairness, accountability, transparency, privacy and regulation, especially where systems influence or automate human decisions. It also names machine ethics, lethal autonomous weapons, arms race dynamics, AI safety and alignment, technological unemployment, AI-enabled misinformation, how to treat AI systems if they have a moral status, superintelligence and existential risk. It adds that healthcare, education, criminal justice and the military carry particularly high stakes.

That list is easier to hold as three groups, shown below. The first is about systems working today. The second is about power and limits. The third is about very capable AI that people may build later.

Bias and fairness: who gets hurt by a model's mistakes

AI systems learn from data, and data can carry old unfairness into new decisions. The article describes a hiring tool at a large company that was dropped because it favoured men: it had learned from ten years of applications that came mostly from men. It also reports that facial recognition systems performed significantly worse on darker-skinned women.

Fairness is harder to pin down than it sounds. The article cites Batya Friedman and Helen Nissenbaum, who sort bias in computer systems into three kinds: existing bias, technical bias and emergent bias. One proposed fix is documentation for the data used in training. It also warns of limits, because the idea of discrimination is itself ambiguous in philosophy and in law.

A court tool called COMPAS shows the problem. It was calibrated so that its error rate was the same across racial groups. Even so, black defendants were almost twice as likely as white defendants to be falsely flagged as high-risk. A system can meet one definition of fair and fail another, so someone has to choose which one counts. That choice is ethical, not only technical.

Transparency, accountability and regulation: who answers, and who writes the rules

Neural networks can make decisions that neither the computer nor its developers can explain. The article says a lack of transparency has been shown to lower user trust. That push has led to calls for explainable AI, and in some places to legal requirements for it. The technical side is covered in mechanistic interpretability for beginners.

Accountability is the sharper edge. The article notes that people tend to treat AI as if it were human-like, which can hide the human negligence or deliberate wrongdoing behind a bad outcome. The EU's AI Act aims to treat AI systems with at least the care of ordinary product liability, which can include audits.

On regulation, the article gives a 2019 Oxford report in which 82% of Americans said robots and AI should be carefully managed. It sets out a real disagreement. Some worry that regulation will slow innovation. Others argue it brings stability that supports innovation in the long term. The EU adopted its Artificial Intelligence Act in June 2024, with a risk-based approach in which some systems are prohibited and others must meet set requirements. In 2021 UNESCO adopted what the article calls the first global standard on the ethics of AI.

Human dignity and weapons: where some say machines should not go

In 1976 Joseph Weizenbaum argued that AI should not replace people in roles that need respect and care, such as a judge, a police officer, a soldier or a therapist. His reason was that these roles need real empathy, and replacing people there threatens human dignity. Pamela McCorduck replied that an impartial computer might be preferable in some conditions. Kaplan and Haenlein stressed in 2019 that such systems are only as good as their training data, so past bias can get formalized.

Weapons are the sharpest case. Under the Convention on Certain Conventional Weapons, states have discussed lethal autonomous weapon systems since 2014. The talks cover international humanitarian law, accountability, possible bans and how much human control is required. Some researchers argue autonomous robots might be more humane. A petition signed by Stephen Hawking, Max Tegmark and others warned that a global arms race would follow if a major military power pushed ahead. For the wider race dynamic, see AI race: what it is and why it worries researchers.

A short history: these questions are older than the models

The article traces moral questions about thinking machines back at least to the Enlightenment, when Leibniz asked whether we should credit intelligence to a mechanism that behaves as if it were a sentient being, and Descartes described what could be seen as an early version of the Turing test. Karel Capek's play R.U.R., which premiered in 1921, is credited with the word "robot". In the 1950s Norbert Wiener wrote about how humans and machines could cooperate, and about the risk of dehumanization.

Isaac Asimov proposed the Three Laws of Robotics to govern intelligent machines, then spent much of his work testing where they break down. That story is told in Asimov's Three Laws of Robotics and AI safety. In 2004 Eliezer Yudkowsky suggested a need to study how to build "Friendly AI". A UK panel in 2010 revised Asimov's laws to say that AI is the responsibility of its makers or its owner or operator. See Friendly AI: what the term means and where it came from.

Where AI ethics meets AI risk

The last items on the list are the far end. The article says many researchers argue that, through an intelligence explosion, a self-improving AI could become so powerful that humans could not stop it from reaching its goals. It reports Nick Bostrom's view that AI could bring about human extinction, and that because a superintelligence need not share human motives, its designers would have to specify its original motivations. It gives his other side too: superintelligence could also help solve disease, poverty and environmental destruction.

The article names researchers who have proposed design strategies for beneficial machines, among them Stuart Russell, Bill Hibbard, Roman Yampolskiy, Shannon Vallor, Steven Umbrello and Luciano Floridi. Two related posts are Can AI tell right from wrong? Machine ethics and its limits and existential risk studies.

Another question on the list is whether AI systems could ever matter morally. In 2020 Shimon Edelman noted how little AI ethics work addressed the possibility of AIs suffering. Thomas Metzinger called in 2018 for a global moratorium until 2050 on work that risked creating conscious AIs. Shulman and Bostrom warn that ignoring digital minds' moral claims could be a moral catastrophe, while putting them ahead of human interests without thought could harm people. Serious people disagree about all of this, and the article takes no side.

A worked example: five questions to ask of any AI system

You can turn the sections above into a check. Take any AI system that makes or shapes a decision about people, such as a tool that ranks job applications.

1. Data: what was it trained on, and who is missing from that data?
2. Fairness: which meaning of fair does it meet, and which does it fail? Remember COMPAS.
3. Explanation: can anyone give a reason for one decision that a person could check?
4. Accountability: can you name a human or an institution who answers for the result, and can a person appeal?
5. Limits: is this a role where, as Weizenbaum argued, a person should stay in charge?

Run the ranking tool through the list. If it was trained on past hires, question 1 flags the risk of repeating a past pattern. If no one can explain a rejection, questions 3 and 4 fail. The same questions get harder, not easier, as systems grow more capable. That is the thread that links everyday AI ethics to the far end of the list.

Frequently asked questions

What is AI ethics in simple terms?

It is the study of how AI should be designed, used and governed so that it treats people fairly and stays under accountable human control. It covers present harms such as bias, and it also reaches to long-term risks.

Is AI ethics the same as AI safety?

No, but they overlap. Wikipedia lists AI safety and alignment as one topic inside the ethics of AI. For the safety field on its own, see AI safety: what the field covers and who works on it.

What is algorithmic bias?

It is when an AI system's results are unfair to some groups because of its data or design. Friedman and Nissenbaum describe three kinds: existing bias, technical bias and emergent bias.

Is there a law on AI?

Some places have one. The EU adopted its Artificial Intelligence Act in June 2024, and UNESCO adopted a global recommendation in 2021. Countries differ, and whether regulation helps or slows innovation is still debated.

Get started

Reading is one way into these questions. Playing through a story is another. Contain ASI is a story strategy game for 1 to 4 players that runs in your browser. It starts in January 2024 with four fictional AI labs in one race, and you try to keep every lab from crossing into recursive self-improvement before 2027. It is inspired by the AI 2027 scenario, and its labs, people and events are all fictional. It is a game, not a forecast, but it lets you explore the far end of this list as fiction.

If you want to keep reading, start with technological unemployment: will AI take our jobs? for the work and power side of the list.

Play Contain ASI: a story strategy game for 1 to 4 players that runs in your browser. Its labs, people and events are all fictional.