language model can write a fluent paragraph about a paper that does not exist. The field calls this hallucination. Psychology has an older and arguably better word, confabulation: producing a plausible account without knowing it is unsupported.

the question

Ji and colleagues (2023) surveyed the term across text generation and kept a useful split: output that contradicts the source it was given (intrinsic), and output that cannot be checked against that source at all (extrinsic) [paraphrase, not a quotation]. The second kind is not always false. It is unsupported, which is a different failure.

the classic findings

Maynez and colleagues (2020) had people read summaries written by neural systems and mark anything not supported by the article. Unsupported content was common, and the usual automatic overlap scores for summaries tracked it poorly [paraphrase, not a quotation]. Lin, Hilton & Evans (2022) wrote TruthfulQA, questions chosen because people often answer them wrongly from popular misconceptions. In their tests the larger models were generally less truthful, because they had learned the misconception more fluently [paraphrase, not a quotation].

A partial counterweight is that models carry some signal about their own reliability. Kadavath and colleagues (2022) found that larger models, asked to estimate the chance that their own answer was right, were reasonably calibrated on many tasks [paraphrase, not a quotation]. Knowing and saying are different steps; see calibration.

Where the fluency comes from. Shannon (1951) estimated how predictable English is by having a person guess a passage one letter at a time. People guess well, because they know the language. A model trained to predict the next piece of text learns the same thing at scale: what usually comes next. What usually comes next and what is true coincide often, which is why it works, and not always, which is this page. The archive copy of Shannon 1948 shows the first, cruder version of the idea.

the human side

Nisbett & Wilson (1977) is the classic case of people explaining their own behaviour fluently and wrongly. In neurology, confabulation names something sharper, in some patients with memory loss, who answer questions about their past with sincere invention. The common thread is that fluency is produced by one process and checking by another, and the second can be missing while the first runs fine.

live debates

Is "hallucination" the right word? Psychiatrists objected early. Østergaard & Nielbo (2023) argue the term is clinically inaccurate, since a model has no perception to misfire, and that it risks stigmatising people who hallucinate; Emsley (2023) prefers "fabrication" and "falsification". Hicks, Humphries & Slater (2024) go further, arguing in Harry Frankfurt's sense that such output is bullshit: produced without regard to truth, rather than a failed attempt at it [paraphrase, not a quotation]. The disagreement is not only verbal. Each label implies a different mechanism and a different fix.

Is it inevitable? Xu, Jain & Kankanhalli (2024) give a formal argument, from computability, that any computable language model will hallucinate on some inputs. Kalai & Vempala (2023) make a statistical one: a model that is calibrated must produce false statements about facts that appear rarely in training, at a rate tied to how many facts appear only once [paraphrase, not a quotation]. Kalai et al. (2025) add an incentive account: if benchmarks grade right-or-wrong and give no credit for abstaining, training rewards confident guessing. Critics answer that a lower bound in theory says little about the rate in practice, which retrieval and abstention can drive down.

methodological critiques

Measuring it. Most measures need a reference. FActScore (Min et al., 2023) splits long answers into atomic facts and checks each against a knowledge source, which makes results depend on the source and on the automatic checker. Semantic entropy (Kuhn, Gal & Farquhar, 2023) (Farquhar et al., 2024) needs no reference: it samples several answers, clusters them by meaning, and flags high disagreement. Its limit is built in: it catches confabulations, answers that vary arbitrarily, and cannot catch errors the model makes consistently. The survey by Huang et al. (2023) separates factuality errors from faithfulness errors (departures from the given input), the distinction Maynez et al. (2020) drew for summaries. A rate reported without saying which kind, against which reference, is not comparable with another.

still open

Whether retrieval of real sources and abstaining when unsure (the reject option) can close most of the gap; whether confidence can be shown to readers in a way they use; and whether people trust a fluent wrong answer more than a hesitant right one, which is a question for automation bias.

Reading list

1. Ji et al. (2023). Survey of Hallucination in Natural Language Generation. ACM Computing Surveys. doi:10.1145/3571730
2. Maynez et al. (2020). On Faithfulness and Factuality in Abstractive Summarization. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. doi:10.18653/v1/2020.acl-main.173
3. Lin, Hilton & Evans (2022). TruthfulQA: Measuring How Models Mimic Human Falsehoods. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). doi:10.18653/v1/2022.acl-long.229
4. Kadavath et al. (2022). Language models (mostly) know what they know. arXiv:2207.05221. arxiv.org/abs/2207.05221
5. Shannon (1951). Prediction and Entropy of Printed English. Bell System Technical Journal. doi:10.1002/j.1538-7305.1951.tb01366.x
6. Nisbett & Wilson (1977). Telling more than we can know: Verbal reports on mental processes. Psychological Review. doi:10.1037/0033-295x.84.3.231
7. Østergaard & Nielbo (2023). False Responses From Artificial Intelligence Models Are Not Hallucinations. Schizophrenia Bulletin. doi:10.1093/schbul/sbad068
8. Emsley (2023). ChatGPT: these are not hallucinations – they’re fabrications and falsifications. Schizophrenia. doi:10.1038/s41537-023-00379-4
9. Hicks, Humphries & Slater (2024). ChatGPT is bullshit. Ethics and Information Technology. doi:10.1007/s10676-024-09775-5
10. Xu, Jain & Kankanhalli (2024). Hallucination is Inevitable: An Innate Limitation of Large Language Models. arXiv:2401.11817. arxiv.org/abs/2401.11817
11. Kalai & Vempala (2023). Calibrated Language Models Must Hallucinate. arXiv:2311.14648. arxiv.org/abs/2311.14648
12. Kalai et al. (2025). Why Language Models Hallucinate. arXiv:2509.04664. arxiv.org/abs/2509.04664
13. Min et al. (2023). FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. doi:10.18653/v1/2023.emnlp-main.741
14. Kuhn, Gal & Farquhar (2023). Semantic Uncertainty: Linguistic Invariances for Uncertainty Estimation in Natural Language Generation. arXiv:2302.09664. arxiv.org/abs/2302.09664
15. Farquhar et al. (2024). Detecting hallucinations in large language models using semantic entropy. Nature. doi:10.1038/s41586-024-07421-0
16. Huang et al. (2023). A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. arXiv:2311.05232. arxiv.org/abs/2311.05232

see also

	calibration — does the confidence match the accuracy?
	sycophancy — a fluent answer shaped by the asker
	interpretability — what the model used, if not the truth
	archive: Shannon, 1948 — approximations to English