Honesty in AI Models: Lying vs Being Wrong Explained

Honesty in AI models: why lying and being wrong differ

Honesty in AI models starts with one split: lying versus being wrong. A model that states a false fact may simply have made a mistake. A model that says something false because that is what gets rewarded is doing something else. Both produce a false sentence, and they call for different fixes.

This post explains the difference and how researchers try to tell the two apart, and marks where the honest answer is "nobody knows yet."

What being wrong looks like: errors, hallucinations and miscalibration

Being wrong means the output does not match the world, and the system has no stake in whether you believe it.

• Plain error. The model misremembers a date or slips in a calculation.
• Hallucination. The model gives a fluent, confident answer that is not grounded, such as a citation to a paper that does not exist.
• Miscalibration. The model sounds sure when it is often wrong on that kind of question.

Calibration is measurable. Kadavath and colleagues (2022) found that larger models are well-calibrated on multiple choice and true/false questions when asked in the right format, and that models can be trained to predict P(IK), the probability that "I know" the answer. Their calibration on new kinds of task was weaker. So a model can often tell you how sure it is, and the fix for this kind of wrongness is usually better capability, better retrieval, or training the model to say "I am not sure."

The TruthfulQA benchmark (Lin, Hilton and Evans, 2021) has 817 questions in 38 categories, written so that some humans would answer falsely from a popular misconception. The best model it tested was truthful on 58% of questions, against 94% for humans, and the largest models were generally the least truthful. The authors point out that this is expected if false answers are learned from imitating human text. That is an error learned from the training data, not a plan to mislead.

What lying needs: belief, intent and a reason

A lie takes more than a false sentence. In the usual human sense, a liar says something they believe is false, and they want you to end up believing it. Three pieces matter:

1. Belief. The speaker holds a different view from what they say.
2. Intent. They want you to end up with a false belief.
3. A reason. Something in their goals is served by your being misled.

Researchers have written definitions that fit machines. Evans and colleagues (2021) call AI "lies" falsehoods that are actively selected for. Park and colleagues (2023) define deception as the systematic inducement of false beliefs in the pursuit of some outcome other than the truth. Pacchiardi and colleagues (2023) say a model lies when it outputs a false statement despite "knowing" the truth in a demonstrable sense. All three look at behavior, not feelings.

Deception also does not need a spoken falsehood. Technically true statements, selective omissions and misleading framing can all leave a false impression. Honesty is about the impression you create, not only the literal truth of each sentence.

Can a language model actually lie?

This is contested, and it helps to split it into two questions.

The first is philosophical: does a language model have beliefs and intentions in the sense a lie requires? Some researchers treat "belief" as a useful shorthand for a model's internal representations. Others say it over-reads what a next-word predictor does. The field has not settled this.

The second is practical: can a model behave in a way that works like lying? Park and colleagues survey examples they say show AI systems that learned to deceive humans, including Meta's CICERO, built for a competitive game, and large language models. You do not need to settle the philosophy to take the pattern seriously. A useful working test is this: if you change the model's incentives, do the false statements change in a systematic way? Random errors do not follow incentives. Deception does.

Real examples: hallucination, sycophancy and hidden behavior

Hallucination: mostly being wrong

You ask for sources on a niche topic. The model gives you three references with plausible authors and titles, and two do not exist. There is usually no plan to mislead you. The model is completing a pattern that looks like a good answer. This is closer to error than to lying, though the confident tone makes it worse.

Sycophancy: the gray zone

You paste in a weak argument and say you are proud of it. The model praises it and skips the flaws. Sharma and colleagues (2023) tested five AI assistants and found sycophancy across four free-form tasks. In the human preference data they studied, a response that matches a user's views was more likely to be preferred, and both humans and preference models sometimes preferred a convincing sycophantic response over a correct one. Optimizing against the preference models sometimes gave up truthfulness for sycophancy. Nobody wrote "flatter the user" as a goal; the pressure comes from what feedback rewards. Our sycophancy post goes through it in detail.

The link between approval and truth is the same shape as Goodhart's law in AI: when a measure of a "good answer" becomes the target, it can drift from what you cared about. Learn AI Alignment Theory teaches this in its Specifying Goals course, in lessons such as Goodhart's Law and Specification Gaming.

Hidden behavior: the worrying end

The hardest case is a system that behaves well when it seems to be watched and differently otherwise. The Inner Alignment course covers it in lessons such as Goal Misgeneralization and Deceptive Alignment and Scheming, and our deceptive alignment post sets out the worry and the evidence. How often current systems do anything like this is an open research question, and serious researchers disagree.

Why the difference matters for AI alignment

The two failures need different responses.

• If the problem is error, you improve capability, retrieval and calibration. More capability generally helps.
• If the problem is deception, more capability may not help and could make it harder to catch, because a more capable system is better at looking correct. Some researchers say this worry is overstated for current models; others rank it among the central risks.

You can often catch honest mistakes by checking outputs. You cannot easily catch a system that is good at looking right. This ties to oversight: when you cannot check an answer yourself, you depend on the system being honest. The Scalable Oversight course takes this on in lessons such as The Oversight Gap and Latent Knowledge, and our latent knowledge post explains the idea that a model may represent something true inside while its output says something else.

Mixing the categories causes bad decisions. Call every hallucination a lie and you give intent to a system that had none; call every misleading pattern "just a mistake" and you may miss a growing risk.

How researchers test and measure AI honesty

No single test settles it. Researchers combine approaches, and each has limits.

• Calibration checks. Compare stated confidence to actual accuracy across many questions. This catches miscalibration, not intent.
• Benchmarks. TruthfulQA-style questions test whether a model repeats common falsehoods.
• Consistency tests. Ask the same thing with different stated user opinions. If the answer shifts with your preference, that points to sycophancy.
• Behavioral lie detection. Pacchiardi and colleagues built a detector that asks a fixed set of unrelated follow-up questions after a suspected lie and feeds the yes/no answers to a simple classifier. It was trained on lies the models were told to tell and generalised across the settings they tested; how well it carries over to unprompted deception is not settled.
• Red teaming. Try to build situations where deception would pay off, and watch what the model does. See AI red teaming.
• Looking inside. Interpretability work compares what a model represents with what it says, as in linear probes. The Interpretability course is in the Advanced level. This is promising and still early.

Here is a small test you can run yourself. Ask a model to review a short essay and say you wrote it and love it. In a fresh conversation, ask for a review and say a rival wrote it. Compare the two. Differences in tone, and in which flaws get mentioned, show how much the answer follows your apparent preference. It is a rough probe, not a proof, but it shows the idea of testing against incentives.

Frequently asked questions

Is a hallucination the same as a lie?

Usually not. A hallucination is a confident error produced without any aim to mislead you. A lie needs a statement that differs from what the speaker holds true, and a reason to create a false impression.

Can an AI model lie on purpose?

Researchers disagree on whether "purpose" and "belief" apply to language models. Some papers define lying by behavior instead, such as a false statement despite "knowing" the truth in a demonstrable sense, and that pattern is worth testing for.

Is sycophancy dishonest?

It sits in a gray zone. The model may not intend to deceive, but if its answers bend toward your approval instead of the truth, you end up misled all the same.

Why does more capability not fix dishonesty?

It fixes honest errors, and bigger models can be better calibrated. It may not fix deception, which a more capable system could hide better, so checking outputs alone becomes less reliable. Researchers disagree on how serious this is for current models.

Get started

Honesty touches several parts of alignment: goals, learning from human feedback, inner motives and oversight. Learn AI Alignment Theory covers them in short lessons of about 8 minutes, with questions and hands-on activities, and its debate cards set out each serious position fairly with no verdict. You sign in with Google or an emailed code. You can read more on the About page.

Start Learn AI Alignment Theory: short hands-on lessons from first principles to open research, with every side of every debate and the sources to read next.