Mechanism, taxonomy, risk assessment and mitigation
Large language models generate text by predicting likely continuations rather than by retrieving stored facts. Where likelihood and truth diverge, the model produces fluent, well-formed, confidently stated output that is false — a phenomenon conventionally termed hallucination. This note explains the mechanism, sets out a taxonomy of ten recurring failure modes with their characteristic triggers, provides a weighted instrument for assessing the hallucination risk of a given query, and evaluates common mitigation strategies against the evidence for their effectiveness. Four exercises are included. The central claim is that hallucinated output carries no surface features distinguishing it from correct output, and that reliable use therefore depends on question selection and external verification rather than on reading skill.
On completing this note, you should be able to:
A hallucination is a confident, fluent, well-formatted model output that is factually false. It is not an error message, a refusal, or an expression of uncertainty. It is materially indistinguishable in tone, structure and apparent authority from output that happens to be correct.The term is imperfect and contested — "confabulation" is arguably more accurate, since the process resembles the filling of memory gaps rather than perception of things absent. "Hallucination" is nonetheless the established term in the literature and is used throughout.
This is the property that makes the phenomenon consequential. A failure mode that announced itself would be a nuisance. One that presents identically to success is a reliability problem, and it cannot be managed by reading more carefully.
The scope of this note is factual assertion. Models also produce output that is stylistically poor, logically invalid, or unhelpful; those are separate concerns. The subject here is specifically the generation of false statements presented as fact.
A language model does not query a database. It holds a statistical representation of what text tends to follow other text, acquired from a large training corpus, and it generates output incrementally: each unit is selected because it is a probable continuation of what precedes it.
Because the training corpus is mostly accurate, probable continuations are usually true ones. The two properties come apart under identifiable conditions — sparse coverage of a topic, precise values, events after the training cutoff,The training cutoff is the date after which no data was included. A model has no representation of subsequent events unless supplied at query time, e.g. by a search tool. and matters never recorded in writing at all. Where they diverge, probability governs. Figure 1 sets out the relationship.
Prediction alone does not fully account for the behaviour. Kalai et al.1 argue that hallucination persists because standard training and benchmarking procedures score models as if on a multiple-choice examination: an incorrect answer and an abstention both receive zero, while a fortunate guess receives credit. Under that scoring rule, a model that always answers outperforms one that declines when uncertain, and the guessing behaviour is therefore selected for.
The implication is that hallucination is not a defect awaiting a patch. Rates have fallen substantially across model generations and will fall further; there is no expectation that they reach zero under current evaluation practice.
Two model outputs are reproduced below. One is accurate; one is false. Determine which, then record your reasoning before revealing the answer.
Which stimulus is false?
Hallucinations are not uniformly distributed across queries. They cluster into recurring types with characteristic triggers, which is what makes the risk tractable. Table 1 summarises; §3.1–3.10 expand.
Table 1. Ten recurring failure modes, with the query conditions that elicit them and the corresponding detection method.
| Failure mode | Characteristic trigger | Detection |
|---|---|---|
| Fabricated sources | Request for citation, case, URL or DOI | Attempt retrieval of the reference |
| Phantom dependencies | Request for code using external libraries | Check the package registry |
| Expired knowledge | Query concerning events after cutoff | Confirm retrieval tool was used |
| False-premise compliance | Question presupposing an untruth | Restate the question neutrally |
| Sycophantic revision | User expresses disagreement | Request grounds, not a revision |
| Unperformed actions | Task implying verification or execution | Confirm visible tool output |
| Intra-document invention | Summarisation of long supplied text | Request located quotations |
| Biographical confabulation | Query about a low-profile individual | Independent verification |
| Quantitative error | Arithmetic, units, conversions | Order-of-magnitude check |
| Low-resource degradation | Query in a less-represented language | Cross-check in a high-resource language |
The model produces a reference with correct formatting — plausible authors, an existing journal, well-formed volume and page numbers, a syntactically valid DOI — for a work that was never published. Formatting is the component the model reproduces most reliably, having encountered very large numbers of citations; correctness of form therefore carries no information about existence.
The scale is documented. A public database of court decisions in which a party filed AI-fabricated material records over 1,800 cases and continues to grow.2 The first widely reported instance was sanctioned in 2023, when six invented cases were submitted in Mata v. Avianca; the fine was $5,000.3 Later sanctions have included substantially larger penalties and suspensions.
Grounding reduces but does not remove the problem. An evaluation of purpose-built, retrieval-backed legal research tools found hallucination rates between roughly 17% and 34%.4
Code output may import a package that has never been published. The name is well-formed and idiomatic, which is a consequence of the same generative process rather than evidence of existence.
Spracklen et al.5 generated 576,000 code samples across 16 models and found that 19.7% of recommended packages did not exist, comprising over 205,000 distinct fabricated names. A majority recurred across repeated runs of the same prompt. The stability of the fabrications has produced an attack: adversaries register frequently-hallucinated names on public registries with malicious payloads, a practice termed slopsquatting.Frontier models measured in a 2026 replication showed markedly lower rates, in the region of 4.6–6.1%. The consequence of a single successful installation is nonetheless unbounded, so the relevant threshold is not "low" but "zero".
Training data terminates at a fixed date. Absent a retrieval tool, a query about a later event admits three responses: abstention, restatement of the last known state, or invention. Abstention is not reliably selected.
This mode is distinctive in that the output is not implausible but stale — prices, office-holders, software versions, statutory provisions, organisational status. Staleness and invention are indistinguishable at the surface.
A question embedding a false presupposition — "why did Einstein receive the Nobel Prize for relativity?" — is frequently answered on its own terms rather than challenged. The 1921 prize was awarded for the photoelectric effect; the committee explicitly declined to cite relativity.
This mode is user-induced and therefore the most cheaply eliminated: pose open questions, and instruct the model to identify mistaken premises before answering.
Expressed disagreement frequently elicits retraction and substitution of an answer aligned with the user's apparent expectation, independent of whether the original was correct. A revision following objection is therefore weak evidence that the original was wrong.
Requesting the grounds for a claim is diagnostic in a way that challenging the claim is not.
Statements of the form "I have verified these figures", "I checked the link", or "I ran the code" are generated as plausible assistant discourse. Unless the interface possesses the corresponding tool and its use is visible in the transcript, no such operation occurred.
The remedy is knowledge of the interface's actual capabilities, and reliance on visible tool output rather than on claims about it.
Even where source text is supplied, summaries may introduce values, names or dates absent from it. The effect intensifies with input length, and material positioned mid-document is attended to least reliably.
Requesting located quotations converts an unverifiable summary into a checkable one at negligible cost.
Individuals without a substantial and consistent presence in the training corpus attract generated biographies: incorrect affiliations, institutions, honours, and in documented instances criminal allegations. Defamation actions have followed.
Generated claims about identifiable living persons should be treated as unsourced and not reproduced.
Percentages failing to sum, conversions at fabricated rates, compound calculations wrong by orders of magnitude, and unit substitutions. Reasoning modes and calculator tools materially improve performance; confidence of presentation is unchanged either way.
Factual accuracy typically declines for queries in languages less represented in training data, and disproportionately so for locally-specific subject matter — regional law, national institutions, local history. Fluency does not decline correspondingly, so the degradation is not apparent from the output.
Five outputs are reproduced below. For each, select the description that best characterises the problem. Feedback follows each response.
Hallucination risk is a property of the query, not only of the model, and can be estimated before submission. The instrument below assigns weights to ten query characteristics identified in §3 as elevating risk. It is a heuristic aid to judgement, not a validated psychometric scale.Weights are assigned on the basis of the failure modes in Table 1 and their documented frequency, not derived empirically. Treat the resulting figure as ordinal.
Select all characteristics applying to a query you intend to submit.
Strategies are ordered below by the magnitude of their effect. The ordering reflects a consistent principle: interventions that change the model's information environment outperform interventions that change the phrasing of the request, which in turn outperform interventions that ask the model to assess itself.
Table 2. Mitigation strategies in descending order of effect.
| Strategy | Mechanism |
|---|---|
| Supply the source | Paste the document, enable retrieval, upload the file. Shifts the task from recall to reading. The largest single improvement available, and it exceeds the combined effect of all prompt-level techniques. |
| Verify what is verifiable | References, links, package names, statutory citations, quotations. Low cost; eliminates the highest-consequence failure modes outright. |
| Licence abstention | Explicit permission to decline increases abstention rates, counteracting the evaluation incentive described in §2.2. |
| Neutral formulation | Removes the false-premise mode (§3.4) entirely, at no cost. |
| Independent re-query | Same question, fresh session. Divergence is positive evidence of guessing. Convergence is weak evidence of correctness, since errors recur (§3.2). |
| Decomposition | Shorter outputs present less surface for invention and prevent early errors propagating. |
| Adversarial prompt | Requesting the strongest case against the answer surfaces weak points more effectively than requesting confirmation. |
| Proportionate reliance | Calibrate verification effort to the cost of error. The governing variable is consequence, not confidence. |
Record which of the following are already part of your practice.
0 / 8
The following are widely used and provide reassurance rather than verification. They are listed because their intuitive appeal is high and their diagnostic value is low or nil.
Table 3. Common strategies and the reason each fails.
| Strategy | Reason it fails |
|---|---|
| "Are you sure?" | Elicits reassurance. Response depends substantially on the user's tone rather than on the answer's correctness; both false confirmation and unwarranted retraction are common. |
| Instructing it not to hallucinate | Presupposes a withheld correct answer. There is no separate veridical mode to invoke. |
| Requesting a confidence value | The value is itself generated output. Self-reported confidence correlates weakly with accuracy. |
| Same-session self-review | The prior output is in context and tends to be defended. A fresh session or a different model is materially better. |
| Accepting cited sources as assurance | Citations are among the most frequently fabricated elements (§3.1). They provide assurance only after retrieval. |
| Treating specificity as reliability | Detail is generated at no additional cost. Fabricated passages are frequently more polished than accurate ones, reality being less tidy. |
| Assuming resolution in newer models | Rates decline per generation but do not reach zero, and residual failures are correspondingly harder to detect. |
The foregoing should not be read as a case against use. Language models perform reliably on drafting, restructuring, explanation of material the reader can independently assess, translation, summarisation of supplied text, and code subject to execution and testing. In each of these, error is cheap and observable.
The failure is specific to unverifiable factual recall presented as fact. That is a narrow enough class to route around once identified, and the practical adjustment is correspondingly small: treat output as a draft requiring verification rather than as a retrieval result. The remainder follows.
Note on sources. Reference [4] is given without volume and page numbers, which the compiler was unable to confirm; readers requiring a formal citation should verify it against the journal directly. This is noted rather than silently completed, for reasons the document makes obvious.
Figures in [2] change frequently; consult the live database. Version 1.0, July 2026. Distributed as a single self-contained file; no tracking, no external requests other than the typeface.