How Task Type Shapes AI Hallucination Risk in Medical Summaries

AI hallucination risk in medical summaries is a deceptively broad phrase. A model asked to draft a discharge-style note from provided clinical text is doing a different job from a model asked to supply literature citations, reconstruct a timeline, or reason through a diagnostic vignette. The evidence looks contradictory only when those tasks are collapsed into one safety metric.

The most useful published numbers do not answer “Does the model hallucinate?” They answer narrower questions: what source material was available, what counted as an unsupported statement, what denominator was used, and whether the error could plausibly change care. On that basis, grounded clinical note generation can look relatively strong, while citation generation and recall-heavy medical tasks can look unsuitable for low-friction deployment.

The Strongest Clinical Note Evidence Is Also the Most Specific

Asgari et al. are useful because they did not treat “summary quality” as a single impressionistic score. Their CREOLA framework evaluated medical text summarization at the sentence level, classified hallucinations, and separated clinically major errors from minor ones. In that setting, hallucinations occurred in about 1.47% of sentences, and 44% of those hallucinations were clinically major [1].

That number is both reassuring and uncomfortable. It is reassuring because a low per-sentence hallucination rate suggests that well-configured systems can produce largely faithful summaries when grounded in source documentation. It is uncomfortable because the denominator is sentences, not patients, notes, or downstream decisions. A low rate can still produce a nontrivial verification task if the misses cluster around medications, diagnoses, test results, allergies, or follow-up instructions.

The same study also reported that the best-performing GPT-4 configurations achieved fewer than one error per note, compared with a human baseline of one error and four omissions per note [1]. That finding deserves to be taken seriously. It complicates the easy story that AI documentation is simply less safe than human documentation. Clinical notes already contain omissions, compression artifacts, and handoff distortions. A grounded model that reduces some of those errors is not merely a novelty.

But it does not follow that the model can be trusted generally. CREOLA’s strength is also its boundary: the model is being judged against available source text. The clinical safety question is therefore operational. Who reviews the generated note? Are reviewers shown the source passages behind each statement? Are major-error categories monitored separately from spelling, style, or redundant phrasing? If a health system deploys summarization as part of ambient documentation or discharge-note drafting, those are the questions that determine whether a 1.47% sentence-level rate is manageable or misleading.

This is where evidence about deployment barriers matters more than broad claims about model intelligence. Workflow design decides whether the clinician is verifying a few highlighted claims or rereading an entire chart to regain confidence. For readers thinking about implementation rather than benchmark comparison, the practical issues overlap with broader barriers in deploying conversational AI in clinical workflows: review time, accountability, integration, and the cost of catching errors after the note has already entered the record.

A Low Aggregate Rate Can Hide the Errors Clinicians Care About

A sentence-level hallucination rate is easy to underread. If most hallucinations were stylistic embellishments, an aggregate rate near 1.5% would carry one kind of implication. When 44% of hallucinations are clinically major, it carries another [1]. The issue is not that every generated note is dangerous. It is that the review process must be designed around the minority of statements that matter most.

Severity grading changes the interpretation of progress. A model can improve its average factuality while still leaving a verification problem concentrated in high-stakes fields. A clinician signing a summary does not experience risk as an average across all sentences. The clinician experiences it as a liability attached to a specific claim: a medication was continued, a symptom resolved, a test was negative, a specialist recommended follow-up, a patient denied a contraindication.

That is why source traceability is not a cosmetic feature. In grounded documentation, a reviewer should be able to move from an output claim to the underlying note, transcript, result, or encounter segment. Without that path, the clinician has to verify the generated text by reconstructing the source record manually. At that point, the model may have saved drafting time while adding a new audit task.

Citation Fabrication Is a Different Failure Mode

Citation generation is often discussed as if it were another form of summarization: the model produces a compact representation of supporting literature. That framing is too forgiving. A citation is not just a sentence with facts in it. It is a pointer to an external object that must exist, be identifiable, and support the claim being made.

Chelli et al. found strikingly different fabrication rates across models when ChatGPT and Bard were evaluated for systematic-review reference accuracy: 18% for GPT-4, 55% for GPT-3.5, and 91% for Bard [3]. Even the lowest figure in that comparison would be unacceptable if references were being relied on without independent verification. The highest figure is not a rough edge around an otherwise useful task; it indicates that the task has been misassigned.

A grounded clinical summary can be checked against supplied source text. A generated reference requires database lookup, metadata matching, and support checking. The model may produce a plausible title, author list, journal, or year because those tokens resemble academic writing, not because it has verified a bibliographic record. Treating that as the same type of hallucination measured in clinical-note summarization obscures the real deployment burden.

The operational consequence is straightforward: literature references generated by a model should not be treated as evidence until they are resolved against trusted bibliographic systems and checked for claim support. That is a stricter standard than asking a clinician to review phrasing in a note. It is closer to source authentication.

Temporal Recall and Lab Understanding Sit Between Notes and Citations

Kim et al. widen the frame by separating medical hallucination across task types, including reasoning, chronological ordering, laboratory data understanding, and other clinical tasks. In their preprint, general clinical reasoning hallucination rates ranged from 0% to 22%, while chronological ordering reached 24.6% and lab data understanding reached 18.7% [2]. The study also reported that 91.8% of surveyed clinicians had encountered medical AI hallucinations and that 84.7% believed such hallucinations could cause direct patient harm [2].

Those survey findings are attention-grabbing, but they need to be weighted carefully. The clinician survey came from a medRxiv preprint and included 70 clinicians [2]. It is a signal worth noticing, especially because clinicians often see failure modes that benchmarks miss, but it should not outrank peer-reviewed, task-specific evaluation when estimating deployment risk.

The task-specific pattern is more important than the headline sentiment. Chronological ordering requires the model to preserve sequence and timing. Lab understanding requires it to map values, reference ranges, clinical context, and sometimes trend direction. These are not merely “summaries” in the sense of compressing text. They are structured recall tasks embedded inside prose.

A model that can produce a fluent hospital-course paragraph may still mishandle whether an event occurred before or after a medication change. It may correctly mention that creatinine changed but misstate the direction, timing, or implication. In a documentation workflow, those errors are harder to catch if the output reads coherently and no source-linked timeline is presented beside it.

The Denominator Decides the Safety Conversation

Hallucination rates become clinically useful only when the denominator is visible. Per sentence, per note, per reference, per case, and per reasoning step are not interchangeable. A 1.5% per-sentence rate in a source-grounded note and an 18% reference-fabrication rate do not describe the same kind of event, and they do not imply the same review process.

• For grounded note generation, the key questions are source coverage, sentence-level faithfulness, clinical severity, and reviewer access to supporting text.
• For citations, the key questions are whether the cited object exists, whether metadata match, and whether the source supports the claim.
• For temporal reconstruction, the key questions are event order, timestamp handling, and whether the system preserves uncertainty.
• For lab interpretation, the key questions are value extraction, trend direction, reference context, and clinical implication.
• For diagnostic reasoning, the key questions are whether conclusions are supported by case facts and whether unsupported differentials are clearly separated from evidence-backed ones.

This is also where regulatory and validation discussions can become too coarse. The number of AI tools entering clinical settings is rising, and the evidence gap around cleared or marketed AI systems remains a separate concern. But a hallucination study is only useful for deployment if its task resembles the intended workflow. A hospital considering AI-assisted documentation needs different evidence from a research team considering AI-assisted literature review. The broader validation problem is better understood as an evidence-quality issue, not simply a model-count issue, as discussed in work on the evidence gap in FDA-cleared AI medical devices.

Model Progress Is Real, but It Does Not Collapse the Task Differences

A mid-2026 snapshot suggests that some grounded summarization tasks have improved. Graffius reported that grounded summarization hallucination rates fell from roughly 1–3% in 2024 to about 0.7–1.5% in top 2025 models [4]. That direction is consistent with the stronger clinical-note results: when the model is supplied with relevant source text and the task is constrained, hallucination can be reduced.

The same trend summary reported a very different pattern for complex reasoning, with hallucination rates rising to 33–51% in newer model classes [4]. This should not be read as a permanent verdict on any one model family; model releases change quickly. It does show why headline performance claims age poorly when they are detached from task design.

The practical lesson is not that newer models are safe or unsafe. It is that “newer” and “larger” are weak substitutes for knowing whether the system is grounded, whether the output is verifiable, and whether the task depends on exact retrieval, temporal order, or external source identity.

How to Calibrate Trust by Task

Grounded clinical note generation is the category where cautious deployment is most defensible, provided the workflow includes clinician review, source traceability, and severity-aware monitoring. CREOLA-style evaluation gives implementers a better template than a generic hallucination rate because it asks whether errors are clinically meaningful, not merely whether generated text sounds plausible.

Citation generation sits at the opposite end. A model can help draft a search strategy, summarize verified papers, or reformat known references. It should not be trusted to invent a bibliography and have that bibliography treated as evidence. The verification layer is not optional; it is the task.

Temporal recall, lab interpretation, and diagnostic reasoning require a middle position with tighter controls than ordinary note drafting. These outputs should expose the facts used to build the conclusion, preserve uncertainty, and allow reviewers to inspect sequence and source. A polished paragraph is not enough when the clinical risk depends on whether events happened in the right order.

The evidence does not support a universal warning against AI medical summaries. It also does not support broad reassurance from low hallucination rates in selected grounded tasks. Trust has to be assigned at the level where the work actually happens: the note, the citation, the timeline, the lab trend, the diagnostic claim. Headline model performance is the wrong unit for clinical trust.

References

1. A framework to assess clinical safety and hallucination rates of LLMs for medical text summarisation. npj Digital Medicine, 2025.
2. Medical Hallucination in Foundation Models and Their Impact on Healthcare. medRxiv, 2025.
3. Hallucination rates and reference accuracy of ChatGPT and Bard for systematic reviews. JMIR, 2024.
4. Are AI Hallucinations Getting Better or Worse? We Analyzed the Data. Scott Graffius, 2026.