AI Hallucination
An instance where an AI system generates false or fabricated information, a nonexistent citation, an incorrect dosage, a misidentified condition, and presents it with the same confident tone it uses for accurate information.
An AI hallucination occurs when a large language model produces output that is factually false, fabricated, or unsupported by its underlying data, while presenting that output with the same fluent confidence as a correct answer. The term comes from the way these models actually work: a language model predicts the next most statistically plausible word given everything that came before, it does not look up facts in a verified database by default. Most of the time that prediction process produces accurate information, because accurate information is usually the statistically common pattern in its training data. But the same mechanism can just as easily produce a plausible-sounding but entirely invented study citation, a fabricated statistic, or a wrong dosage, with no internal signal that distinguishes the fabrication from a genuine fact.
In a medical context, hallucination is especially consequential because the output often looks like rigor: a specific number, a named study, a precise-sounding clinical detail. A 2023 study published in Cureus found that when an early chatbot model was asked to generate short medical papers with supporting citations, roughly half the citations it produced did not exist at all, and most of the real citations it did produce misrepresented their actual source. Newer models, particularly those using retrieval-augmented generation, have meaningfully reduced these rates by grounding responses in verified source material rather than generating from trained parameters alone, but the failure mode has not been eliminated.
Asked to explain a drug interaction, an AI model might state a specific, confident-sounding dosage adjustment and cite a clinical guideline that does not actually contain that recommendation, or does not exist at all. The answer reads identically to a correct one; only checking the cited source against the actual guideline reveals the fabrication.