AI Hallucinations in Healthcare: What the Research Actually Shows

One of the most frequently cited pieces of evidence on this problem is a 2023 study published in Cureus, in which researchers asked an early chatbot model to generate short medical papers with supporting citations. Roughly half the references it produced did not exist at all, invented journal names, invented authors, invented DOIs, attached to a plausible-sounding claim. Most of the remaining citations that were real turned out to misrepresent what the source actually said. Only a small fraction were both real and accurately described. Nothing about the output looked uncertain. That is the core problem with AI hallucination in a medical context: it is not hedged, half-remembered, or flagged as a guess. It reads exactly like a correct answer.

What "Hallucination" Means in a Medical Context

Outside medicine, an AI hallucination might mean a wrong date or a misattributed quote, annoying, but low stakes. In a medical context the same failure mode attaches to information people act on: a fabricated study used to justify a supplement claim, an incorrect drug dosage stated with total confidence, a rare condition misidentified as a common one, an imaging description that invents a finding that isn't there. The defining feature isn't that the AI is wrong. Every information source is sometimes wrong. It's that a large language model has no internal signal that distinguishes a fact it retrieved from a plausible-sounding pattern it generated. Both come out in the same fluent, confident voice.

What the Research Actually Shows

• Citation fabrication is a real, measured problem. The Cureus study above is one of several finding that early general-purpose chatbot models frequently invented references when asked to support medical claims, a distinct and especially dangerous failure mode because a fake citation looks like rigor.
• Structured, exam-style medical questions are where models perform best. Google's Med-PaLM 2 and successor medical models have reported expert-level accuracy on USMLE-style licensing exam questions, in the high 80s percent range, which reflects strong performance on well-documented, guideline-based knowledge with a single correct answer.
• Open-ended, real-world questions are where performance drops. The gap between exam-style accuracy and accuracy on ambiguous, multi-symptom, real-patient-style questions is consistently the weak point across independent evaluations, precisely the format most people actually use when they ask an AI about their own health.
• Hallucination rates are falling with each model generation, largely due to retrieval-augmented generation and more aggressive fine-tuning against fabrication, but falling from a real and previously high baseline is not the same as solved.

Where the Risk Concentrates

• Rare and atypical conditions, where training data is thin and a model is more likely to pattern-match to a common look-alike condition instead.
• Drug interactions and dosage calculations, where the cost of a confidently wrong number is immediate and where correct answers depend on variables (weight, kidney function, other medications) a chat interface often doesn't have.
• Reading and interpreting imaging descriptions, where a model can generate a plausible-sounding but invented finding rather than accurately summarizing what a radiology report actually says.
• Anything requiring current, local, or fast-changing guidance, updated screening ages, new drug approvals, revised dosing guidance, since a model's knowledge has a cutoff and it doesn't always signal that clearly.

Where the Risk Is Lower

• Explaining a common, well-documented condition in plain language.
• Summarizing an established clinical guideline that hasn't changed in years.
• Generating a list of questions to bring to a doctor's appointment.
• Defining a medical term or translating jargon from a lab report or discharge summary.

How the Platforms Are Addressing It

Grounding is the main lever the industry is pulling. Retrieval-augmented generation (RAG) forces a model to pull from a specific, verifiable source, a medical database, a set of clinical guidelines, a curated literature index, rather than generating an answer purely from its trained parameters. Perplexity Health's whole product is built around citation-first answers for this reason. Verily's model pairs AI output with clinician review rather than shipping the AI's answer directly to the patient. Several platforms now also add explicit confidence framing or refuse to answer dosage-specific questions outright, trading some helpfulness for safety.

Practical Guidance: How to Protect Yourself

• Ask the AI to cite its source, then actually check that the source exists and says what the AI claims.
• Treat any specific number, dosage, lab reference range, drug interaction, with more skepticism than a general explanation.
• Don't use an AI's answer as the final word on a rare or unusual symptom pattern; use it to generate better questions, not a diagnosis.
• Re-verify anything time-sensitive (a new drug approval, an updated screening guideline) against a current primary source rather than trusting a model's training-data snapshot.

The Bigger Picture

Hallucination rates in medical AI are declining fast, retrieval grounding and better fine-tuning have measurably improved newer models over their 2023-era predecessors. The open question isn't whether the trend line is improving; it clearly is. It's whether it improves fast enough to keep pace with how quickly these tools are being pulled into higher-stakes use cases. See our companion assessment of what today's AI health platforms can and can't do for how each major platform is handling that tradeoff, and Age Better Today's practical guide to using AI tools for healthy aging for the day-to-day consumer angle.