AI in Healthcare Has an Evidence Problem

The use of AI in healthcare has moved faster than the evidence used to justify it. That is not a complaint about innovation; it is a problem of claims. A model can detect a pattern in an image, summarize a note, flag a patient for review, or prioritize a worklist. Those are different acts with different consequences. They should not all be allowed to borrow the same confidence from a validation score.

In April 2026, a Nature Medicine editorial put the issue plainly: “evidence that AI tools create value for patients, providers or health systems remains scarce.” It called for evidence standards that match the claims being made about medical AI rather than treating technical performance as a stand-in for clinical value.[1] The concern becomes harder to dismiss when placed beside the 2025 JAMA audit of FDA-cleared AI and machine learning devices: among 691 devices analyzed, only 6 cited a randomized clinical trial, only 3 reported patient health outcomes, 46.7% of FDA summary documents did not describe study design, and 53.3% omitted sample size.[2]

That is the center of the evidence problem. It is not that every AI tool is unsafe, useless, or overhyped. It is that adoption, authorization, accuracy, and value are being discussed as though they were interchangeable.

Adoption Has Outrun Evaluation

By 2026, 81% of physicians reported using some form of health AI, up from 38% in 2023, according to the AMA.[3] That figure should be read carefully. “Health AI” can include administrative, documentation, communication, and clinical tools, not only diagnostic algorithms. Still, it marks a clear shift: AI is no longer a speculative layer outside the health system. It is already inside clinical work, revenue cycles, inboxes, imaging queues, and procurement plans.

Regulatory volume tells the same story from another angle. FDA-authorized AI and machine learning medical devices reached 1,451 cumulatively, with 295 authorized in 2025 alone, a record year. Most moved through the 510(k) pathway: 97% of listed devices used that route, with a median clearance time of 142 days.[4] For a closer look at those authorization trends, see AI medical technology authorizations.

Market estimates add scale, though not much precision. A $36.7 billion figure signals investor and institutional momentum, but market sizing depends heavily on what counts as healthcare AI. A hospital scribe, a radiology triage algorithm, a payer automation system, and an oncology decision-support model may all appear under the same umbrella while carrying very different clinical risks.

The useful question is therefore not whether AI is being used. It is what kind of evidence has been required before use, what claim that evidence actually supports, and who is left to manage the gap when a tool behaves differently in routine care.

What the JAMA Audit Actually Shows

The JAMA audit matters because it does not rely on a generalized discomfort with AI. It looks at the documentary trail behind cleared devices. The finding that only 6 of 691 devices cited a randomized clinical trial does not mean all other devices failed clinically. It means the public evidence base rarely tested whether using the tool changed outcomes under conditions that resemble care delivery.[2]

The patient-outcomes signal is even thinner. Only 3 devices reported patient health outcomes.[2] That distinction is critical. A device can improve sensitivity, reduce a turnaround interval, or classify a scan correctly in a retrospective dataset without showing that patients are diagnosed earlier, treated more appropriately, harmed less often, or spared unnecessary workups.

The transparency gaps are not clerical details. If 46.7% of summaries do not describe study design and 53.3% omit sample size, a hospital cannot easily tell whether evidence came from a retrospective single-site dataset, a multi-site prospective study, an enriched sample, or a setting that looks nothing like its own patient population.[2] That makes local governance harder, not just academic review harder.

A separate 2025 npj Digital Medicine analysis also found limited transparency around AI and machine learning model characteristics across 1,016 analyzed FDA authorizations.[5] That is a different but related problem. Even when a device is authorized, users may not have enough information to judge model inputs, update practices, population fit, or performance boundaries.

A clearance summary is not a deployment manual. It may be enough to show that a device met a regulatory threshold for a stated intended use. It is not, by itself, proof that the device will create value in a particular emergency department, imaging service, primary care network, or ICU.

Technical Performance Is Not the Same as Clinical Value

The most common slippage in AI evaluation is moving too quickly from “the model performed well” to “the health system should buy it.” Technical performance is necessary for many clinical AI tools, but it is not the end of the evidence chain.

Lung cancer screening illustrates the gap. Reported AI sensitivity ranged from 56.4% to 95.7%, compared with radiologists at 23.2% to 76% in the cited performance contrast.[6] Those numbers are interesting, especially if a tool consistently detects cases that humans miss. But higher retrospective sensitivity does not answer several operational questions: whether false positives rise, whether radiologists change behavior, whether downstream testing increases, whether time to diagnosis improves, or whether patients ultimately benefit.

This is not a theoretical nicety. A triage tool that moves suspected cases to the top of a queue can help if it shortens clinically meaningful delays. It can also displace other patients, create new review obligations, and generate alerts that busy teams learn to ignore. The same sensitivity figure can look very different once it enters staffing constraints, reading room norms, malpractice anxiety, and patient follow-up capacity.

Retrospective accuracy also depends on the dataset. If cases are selected, labels are cleaner than real-world documentation, or the population differs from the deployment site, the reported performance may not travel. External validation helps, but even external validation is still not the same as showing that care changed for the better.

The right interpretation is narrow: a strong retrospective result can justify further evaluation, pilot deployment under supervision, or a prospective study. It should not be treated as proof of improved survival, reduced workload, or lower cost unless those outcomes were actually studied.

FDA Authorization Answers a Different Question

FDA authorization is meaningful. It can define intended use, impose regulatory review, and require evidence that a device meets a threshold for safety and effectiveness within that regulatory frame. The problem begins when authorization is converted into a broader claim: that the tool improves outcomes, saves clinicians time, reduces costs, or performs well for every site that buys it.

The dominance of the 510(k) pathway matters here because that route often depends on substantial equivalence to a predicate device rather than the kind of prospective comparative study many clinicians assume sits behind a medical AI product. The 97% figure does not make those authorizations invalid; it clarifies what they should not be asked to prove.[4]

Hospitals sometimes blur that boundary because procurement needs a yes-or-no decision. A cleared tool comes with a vendor deck, an implementation timeline, and a projected return. The evidence review then becomes a late-stage hurdle instead of an early question: What exactly are we buying this tool to improve, and what evidence would count as improvement?

That question should be asked before the pilot, not after alert rates rise or clinicians begin building workarounds.

The Evidence Burden Should Rise With the Claim

Not every AI use case needs a randomized trial. A low-risk administrative tool that drafts a message, organizes a queue, or reduces duplicate data entry can reasonably be judged by usability, error rates, privacy controls, and measured time savings. The standard changes when the tool influences diagnosis, triage, treatment selection, discharge decisions, or escalation of care.

• Low-risk workflow tools should show that they reduce burden without introducing unacceptable errors, privacy risks, or hidden review work.
• Documentation and communication tools should be evaluated for accuracy, clinician editing burden, patient safety risk, and whether time saved is real rather than shifted.
• Triage and prioritization tools should show prospective performance in the intended workflow, including false positives, false negatives, alert burden, and downstream effects.
• Diagnostic and treatment-influencing tools should require stronger validation, transparent study design, relevant comparators, and outcome measures aligned with the claim.
• Tools marketed as improving patient outcomes, staffing, or cost should be required to measure those outcomes directly or state that the claim remains unproven.

This proportional approach is close to the governance standard called for in the Nature Medicine editorial: the evidence required should match the value claim and the risk created by deployment.[1] It also gives health systems a more practical way to evaluate AI than a single pass-fail demand for randomized trials in every case.

The harder cases are the middle ones. A sepsis prediction tool, for example, may look like decision support rather than a diagnostic device, but it can change who receives attention, labs, antibiotics, or ICU review. A documentation assistant may appear low risk, but if it fabricates or normalizes an incorrect clinical statement, the burden shifts to clinicians and patients. Evidence standards should follow the consequence, not the product category.

Some AI Tools Have Better Evidence Than the Field Overall

The evidence gap is not uniform. Ambient documentation tools have attracted more direct study of clinician experience, documentation burden, and time use than many higher-risk clinical AI categories. Some sepsis and workflow applications have also been evaluated more seriously in selected settings. That matters because a blanket dismissal of healthcare AI would miss where the evidence is improving.

It also prevents the wrong lesson. The problem is not that AI cannot create value in medicine. The problem is that the field too often allows the strongest examples to lend credibility to weaker ones. A well-evaluated scribe does not validate an imaging algorithm. A prospective study in one hospital does not prove performance in another. A workflow gain does not establish a patient-outcome benefit.

Readers looking for a broader map of where evidence is stronger and weaker can compare this article with where AI delivers in healthcare and where it does not. The distinction is important for governance: evidence should travel only as far as the study design allows.

Post-Market Signals Are Too Sparse to Carry the Burden

Post-market data should help reveal whether AI tools continue to perform safely after deployment. The available safety signals are limited. In the cited post-market data, only 5.2% of AI devices had any adverse event report, and 5.8% had ever been recalled, mostly for software bugs.[7]

Those figures should not be overread. Low reporting could mean few problems. It could also reflect under-detection, uncertain attribution, weak reporting habits, or difficulty recognizing when an AI recommendation contributed to harm. A missed diagnosis, delayed escalation, or unnecessary test may not be traced back to a model unless the health system is actively looking.

For AI, post-market monitoring has to be more than a passive complaint channel. It should include performance drift checks, subgroup performance review, alert override patterns, false-positive and false-negative tracking, incident review, and a clear process for pausing or changing use when performance no longer matches the original claim.

What a Serious Evaluation Standard Looks Like

A serious evaluation standard begins by separating claims. “FDA-cleared,” “accurate,” “clinician-friendly,” “workflow-improving,” “cost-saving,” and “outcome-improving” are not synonyms. Each one needs its own evidence.

For clinicians and administrators, the practical version is straightforward: identify the claim, identify the affected workflow, identify who bears the risk, and then ask whether the evidence directly measures that claim in a setting close enough to the one where the tool will be used. A more operational checklist belongs in an implementation framework, such as how to evaluate AI tools in clinical practice.

The uncomfortable part is that this process slows some purchases. It may also prevent some launches. That is not a failure of innovation; it is a normal response when a technology makes claims about patient care, clinical labor, or health system value without evidence that reaches those endpoints.

Healthcare AI should be judged by claims matched to evidence. FDA clearance, retrospective performance, prospective validation, clinical outcomes, workflow impact, and post-market monitoring need to remain visibly separate. When a tool claims only to route messages faster, the evidence burden can be modest. When it claims to improve diagnosis, reduce harm, or lower system costs, the burden rises accordingly.

References

1. Show us the evidence for the value of medical AI, Nature Medicine, April 2026.
2. JAMA evidence audit of FDA-cleared AI/ML devices, JAMA, 2025.
3. AMA physician survey on health AI adoption, American Medical Association, 2026.
4. FDA AI/ML medical device authorization data, U.S. Food and Drug Administration.
5. npj Digital Medicine study of transparency in FDA AI/ML authorizations, npj Digital Medicine, 2025.
6. Lung cancer screening AI and radiologist sensitivity performance contrast.
7. Post-market safety data for AI medical devices.