How Much Clinical Proof Do FDA-Cleared AI Devices Actually Have?

A committee packet lands on the agenda with a familiar phrase near the top: FDA-cleared AI. For AI use in healthcare, that label matters. It tells the hospital that the device has passed through a regulatory gate. It does not tell the radiology chair, the procurement analyst, or the quality officer whether the tool improves diagnosis, changes treatment, reduces harm, saves clinician time, or works on the hospital’s own patients.

That distinction is getting harder to keep visible because the clearance pace is now extraordinary. The FDA maintains a public database of AI-enabled medical devices, and IntuitionLabs’ aggregation of that database counted 1,451 authorized AI/ML devices by the end of 2025; Innolitics reported 295 AI/ML medical device 510(k) clearances in 2025 alone, a record annual total.[1][2][3] The volume creates a reasonable impression that the field is maturing. The evidence record says something narrower.

A JAMA Network Open study of FDA-approved AI-enabled medical devices found that fewer than 2% were supported by randomized clinical trial evidence and less than 1% reported patient health outcomes.[4] The same study is cited as finding that 46.7% of FDA 510(k) summaries omitted a study design description and 53.3% omitted sample size.[4] Those reporting-gap figures deserve a verification caveat: the full JAMA article was not accessible during reporting, so the specific 510(k) summary percentages are being treated here through secondary citation by IntuitionLabs, not independent extraction from the journal text.[2][4]

Even with that caveat, the direction of travel is not subtle. FDA authorization has scaled much faster than public, decision-grade clinical evidence. That does not make the devices useless. It does mean the words “FDA-cleared” should not be allowed to stand in for an evidence review.

What FDA Clearance Actually Signals

FDA clearance is a legal and regulatory status. For many AI medical devices, especially those cleared through the 510(k) pathway, the central question is whether the new device is substantially equivalent to a predicate device that is already legally marketed. It is not the same question as: has this product been prospectively tested in this hospital’s workflow, against current standard care, with patient outcomes measured?

In a purchasing discussion, those categories often collapse into one another. A cleared triage tool may have evidence of technical performance on an image set. It may have analytical validation showing that the algorithm detects a pattern with a stated sensitivity or specificity. It may have clinical validation showing performance in a relevant patient population. It may have a workflow study showing faster notification or reduced time to review. It may have outcomes evidence showing that patients received better care because of the tool. These are not interchangeable forms of proof.

Randomized trials are not the only legitimate evidence for every AI device. Some tools are closer to measurement, triage, or workflow support than to a therapeutic intervention. Prospective validation, external validation, reader studies, silent-mode evaluations, and workflow studies can all be meaningful when they match the claim. The problem is not that every device lacks an RCT. The problem is that basic information about what was tested, how, on whom, and with what endpoint is often not visible enough for adoption decisions.

The Evidence Gap Is Not Just Academic

A missing study design field is not a scholarly inconvenience. It changes the work a hospital has to do before purchase. If a 510(k) summary does not say whether evidence came from a retrospective dataset, a multi-site external validation, a reader study, or a prospective workflow deployment, the buyer cannot judge how close the evidence is to the intended use.

A missing sample size is just as operational. It leaves the analyst unable to tell whether performance rests on a broad enough test set, whether rare subgroups were meaningfully represented, or whether confidence around the reported result is too wide for clinical comfort. The radiologist or cardiologist who will inherit the tool’s false positives and false negatives needs those details before the contract is signed, not after the dashboard goes live.

The patient-outcomes finding is the sharpest version of the same issue. Less than 1% of devices in the JAMA analysis reported patient health outcomes.[4] That does not mean the other devices have no value. A tool that shortens time to review a critical image may matter even before mortality or morbidity data exist. But if the sales claim shifts from detection or prioritization to better patient care, the evidentiary burden shifts with it.

Market momentum can make this look less urgent than it is. Menlo Ventures reported that healthcare-specific generative AI spending reached $1.4 billion in 2025, nearly tripling 2024 investment, and that health systems accounted for 75% of that spending.[7] That figure is useful as a pressure gauge. It is not evidence that a given cleared device is clinically valid.

Why 510(k) Dominance Matters for AI

The 510(k) pathway is doing most of the work in this market. Available analyses report that 97% of clearances used 510(k), which matters because the pathway is built around substantial equivalence rather than a fresh demonstration of clinical benefit for every new device. That can be efficient and appropriate for many incremental products. It becomes more complicated when the product is software whose performance may depend on training data, deployment setting, workflow integration, and post-market updates.

Substantial equivalence is not a trick word; it is a regulatory tool. The issue is what happens when the phrase is moved from an FDA review context into a hospital adoption context. A device can be substantially equivalent to a predicate and still lack published evidence that it improves outcomes. It can perform acceptably in the submitted testing package and still need local validation before clinicians rely on it. It can be cleared for a specific intended use and still be marketed in a way that tempts people to infer broader clinical value.

AI also makes the predicate relationship less intuitive. A software tool cleared in 2025 may be linked to a predicate from an earlier technical generation. The new product may use different data sources, model architectures, user interfaces, alert thresholds, or workflow assumptions. None of that automatically invalidates the clearance. It does mean the adoption committee should ask exactly which features were treated as equivalent and which claims were actually tested.

Post-market change is the next stress point. Predetermined Change Control Plans are intended to define, in advance, how certain algorithm changes can be made while preserving safety and effectiveness oversight. Innolitics reported that only 10% of 2025 AI/ML device clearances included PCCPs.[3] For a hospital, that raises a practical question: if the model changes after go-live, who is notified, what validation is repeated, and what performance drift triggers intervention?

The Clearance Landscape Is Heavily Skewed Toward Radiology

AI clearance is not evenly distributed across medicine. IntuitionLabs’ analysis of the FDA database found that 76% of cleared AI devices are in radiology, with cardiology a distant second at roughly 9%.[2] That distribution is not surprising. Radiology has digital inputs, large image archives, relatively clear detection and triage tasks, and established performance metrics. It is also where many hospitals first encounter FDA-cleared AI in a form that looks operationally deployable.

The concentration matters because it can distort general impressions of AI use in healthcare. A clearance boom led by imaging tools does not mean the same level of regulatory maturity exists in every specialty or use case. A stroke triage algorithm, a cardiac measurement aid, a sepsis prediction model, and a generative documentation assistant create different evidence questions. Counting them under one AI umbrella can hide more than it reveals.

Radiology also shows why workflow evidence cannot be an afterthought. A detection or prioritization tool may shift who looks first, how urgently a study is escalated, how many false positives interrupt the reading queue, and how clinicians respond to discordance between the model and the human reader. Technical accuracy is only one part of that chain.

Newer AI Forms Are Reaching the Regulatory Frontier

The frontier is not standing still. MedTech Dive’s tracker reported that Aidoc CARE1, described as the first foundation-model-powered clinical AI, received FDA clearance in February 2025.[5] Innolitics also noted RecovryAI’s FDA breakthrough device designation for generative AI in drug repurposing.[3] These are not just novelty items. They signal that regulators, manufacturers, and hospitals are moving from task-specific algorithms toward systems with broader technical foundations and more complex update questions.

Foundation models make evidence review more demanding, not less. A narrower model can often be evaluated around a bounded input, output, and intended-use claim. A broader model may support multiple downstream tasks or be integrated into larger software systems. The buyer then needs to know which part of the system was cleared, which claims were reviewed, what data supported each claim, and what happens when the model or its surrounding workflow changes.

Breakthrough designation should be read with the same discipline. It can signal that FDA sees a device as addressing an important need and eligible for prioritized interaction. It is not the same thing as proof of clinical effectiveness, and it is not a substitute for the evidence package a hospital needs before adoption.

Weak Evidence Can Become a Fairness Problem

Evidence gaps are sometimes discussed as if they only burden administrators. They can also affect patients unevenly. A review in npj Digital Medicine summarized the well-known Obermeyer cost-proxy case, in which using healthcare cost as a proxy for illness severity underestimated Black patients’ needs by 26.3%; when the algorithm was recalibrated with direct health indicators, enrollment of high-risk Black patients nearly tripled from 17.7% to 46.5%.[6]

That case was not about every FDA-cleared AI device, and it should not be used as if it were. Its value is more specific: it shows how a convenient proxy can embed inequity even when a model appears to be performing a reasonable administrative task. If a device summary does not disclose enough about data, endpoints, subgroup performance, and intended workflow, the adopting institution has less ability to see where similar problems might enter.

The same npj Digital Medicine review reported broader bias concerns, including that 50% of healthcare AI models in a systematic PRISMA review showed high risk of bias and only 1 in 5 had low risk.[6] That finding should not be pasted onto every device as a verdict. It should be treated as a reason to ask for subgroup analysis, external validation, and monitoring before a model is placed into a clinical queue.

The Evidence Audit After “FDA-Cleared”

Once a device is identified as FDA-cleared, the next step is not to dismiss it or approve it. The next step is to separate the regulatory signal from the adoption evidence. A hospital can do that with a short but firm audit that asks for information in the form clinicians and procurement staff can actually use.

• Confirm the exact cleared indication: identify the intended use, user, patient population, input data, output, and clinical setting reviewed by FDA.
• Map the regulatory pathway: determine whether the device was cleared through 510(k), de novo, or PMA, and identify the predicate if 510(k) was used.
• Request the full evidence package: do not rely only on a public summary if it omits study design, sample size, comparator, endpoint, or test population.
• Classify the evidence type: distinguish analytical performance, retrospective validation, external validation, prospective workflow study, randomized trial, and patient outcome evidence.
• Check the deployment match: compare the study population, scanner or device inputs, EHR environment, staffing model, and clinical workflow with the local setting.
• Inspect subgroup and bias reporting: look for performance by race, ethnicity, sex, age, site, disease severity, and other clinically relevant groups where appropriate.
• Define monitoring before go-live: assign responsibility for false positives, false negatives, override review, drift checks, update review, and incident escalation.

The audit should be proportionate. A low-risk measurement aid does not need the same evidence package as a tool that changes triage priority for a life-threatening condition. But proportionality is not the same as vagueness. If the manufacturer claims the tool reduces time to diagnosis, ask for time-to-diagnosis evidence. If the claim is improved outcomes, ask for outcomes. If the claim is generalizability, ask where it was externally validated.

Local validation is not a ceremonial step. It is where the hospital learns whether the model’s alerts arrive at the right point in the workflow, whether clinicians trust them enough to act, whether alert volume is tolerable, and whether performance shifts across patient groups. A silent trial, limited pilot, or phased rollout can reveal problems that a clearance summary was never designed to answer.

Questions That Should Not Be Left to the Contract Stage

• What exact version of the model was cleared, and is it the version being sold?
• Were test data independent from training and tuning data?
• Was performance evaluated at external sites, or only in data similar to development data?
• What happens when the model is updated, and does the clearance include a PCCP?
• Who reviews failures, overrides, and clinician complaints after deployment?
• What evidence would trigger renewal, expansion, retraining, suspension, or removal?

These questions are not hostile to innovation. They keep the adoption burden from sliding quietly onto the clinicians who must use the tool and the patients who experience its errors. They also protect the organization from buying a regulatory status when what it needs is a working clinical intervention.

A Practical Reading of the Label

FDA clearance should remain an important screen. For many medical devices, it is a necessary signal that the product has passed regulatory review for a specified use. It is also a weak substitute for clinical validity when the public record does not disclose basic evidence elements or when the claim being made in the hospital exceeds the claim that was actually tested.

The disciplined position is simple enough for a committee packet: FDA-cleared means authorized, not clinically proven. Before adoption, clinicians and buyers still need an independent evidence audit covering study design, sample size, endpoints, external validation, subgroup performance, update controls, and post-market monitoring. If those fields are blank, the gap does not disappear. It becomes local work.

References

1. FDA Artificial Intelligence-Enabled Medical Devices database — FDA
2. FDA's AI Medical Device List: Stats, Trends & Regulation — IntuitionLabs
3. 2025 Year in Review: AI/ML Medical Device 510(k) Clearances — Innolitics
4. Generalizability of FDA-Approved AI-Enabled Medical Devices — JAMA Network Open
5. AI in medtech is booming. Track new devices here — MedTech Dive
6. Bias recognition and mitigation strategies in artificial intelligence healthcare applications — npj Digital Medicine
7. 2025: The State of AI in Healthcare — Menlo Ventures