The arithmetic behind AI medical imaging is uncomfortable before any debate about algorithms begins. By the end of 2025, radiology accounted for 1,104 of 1,451 cumulative FDA-authorized AI/ML medical devices, or 76% of the total. In 2025 alone, 295 new AI/ML devices were cleared or authorized, and radiology still held a 71.5% share.[1] The specialty has become the main proving ground for medical AI.
That scale can make clearance sound like accumulated clinical proof. It is not. In a JAMA Network Open analysis of 691 AI/ML-enabled medical devices through July 2023, fewer than 2% had randomized clinical trial support. In the same evidence set, 46.7% of FDA decision summaries omitted study design, and 53.3% omitted sample size.[2] Across a broader group of 903 devices through August 2024, only 55.9% had publicly available clinical performance data at clearance.[2]
Those denominators matter. The figure about randomized trials does not describe every device cleared through 2025; it comes from 691 devices analyzed through July 2023. The public performance-data figure comes from 903 devices through August 2024. The 1,104 radiology-related devices belong to the cumulative FDA-authorized AI/ML count through the end of 2025. Blending those numbers into one generic claim would be sloppy. Read correctly, they point in the same direction: market access for imaging AI has expanded much faster than public, independently reviewable clinical evidence.
For a radiology department, the missing fields are not academic niceties. Study design tells a quality committee whether performance came from retrospective enrichment, prospective collection, or something else entirely. Sample size tells the PACS and modality teams whether the validation set resembles routine clinical volume or a narrow test collection. Subgroup reporting tells clinicians whether performance was examined across the patients who will actually be scanned. Without those details, “FDA-cleared” answers a legal question but leaves the operational question unresolved.
Clearance Answers a Narrower Question Than Clinicians Often Need
Most AI/ML medical devices have reached the market through the 510(k) pathway. In 2025, 97% of AI/ML devices used that route.[1] The practical point is simple: 510(k) clearance rests on substantial equivalence to a predicate device for a specified intended use. It does not require a new randomized trial showing that the device improves diagnosis, outcomes, turnaround time, reporting quality, patient management, or safety in the hospital that is about to buy it.
That distinction is not a criticism of the FDA for failing to run a hospital’s implementation program. It is a boundary condition. Clearance permits marketing for the cleared intended use. It does not tell a stroke service whether the tool will page the right team at the right time, whether technologists will change acquisition behavior, whether false positives will fatigue the reading room, or whether a local patient mix will expose failure modes not visible in the public summary.
This is where procurement language often gets dangerous. “FDA-cleared” is heard by administrators as a safety signal, by clinicians as a performance implication, by vendors as market legitimacy, and by implementation teams as another interface to make behave inside an already crowded workflow. Those are not the same interpretations. If the evidence package is thin or inaccessible, the burden shifts downstream to radiologists, technologists, IT analysts, and quality committees after purchase.
What Is Often Missing From Public Evidence
The most useful evidence gap is not “AI is unproven” in the abstract. It is the smaller, more concrete problem that public materials often omit the information a clinical team would use to judge transferability. In the JAMA Network Open analysis, nearly half of 510(k) summaries omitted study design and more than half omitted sample size.[2] If a summary does not say how a device was tested or how many cases supported the result, a hospital cannot tell whether reported performance was robust, fragile, or simply not interpretable from public information.
| Evidence item | Why it matters before deployment |
|---|---|
| Study design | Separates retrospective testing, prospective validation, enrichment strategies, and workflow studies. |
| Sample size | Shows whether performance estimates rest on enough cases to be credible for the intended use. |
| Patient and scanner context | Helps determine whether validation resembles the hospital’s population, modalities, sites, and acquisition patterns. |
| Subgroup performance | Reveals whether aggregate performance hides weaker results by sex, age, race or ethnicity, or other relevant clinical groups. |
| Workflow endpoint | Clarifies whether the device improves a clinically meaningful step or only performs well against a test label. |
Subgroup reporting is particularly thin. Only 28.7% of studies reported separate performance by sex, 23.2% by age, and 15.8% by race or ethnicity.[2] Those numbers should not be stretched into a claim that every device performs inequitably. They show something narrower and still important: for many devices, the public record does not allow clinicians to evaluate whether performance is consistent across groups.
The absence of public evidence is also not the same as proof that no evidence exists. A vendor may have unpublished analyses, proprietary test results, or customer validation data under confidentiality restrictions. But “data may exist somewhere” is not a standard a clinical governance committee can rely on. If the people expected to use the device cannot review the design, population, endpoint, and failure modes, they cannot independently evaluate the claim.
Randomized Trials Are Scarce, but They Are Not the Only Useful Evidence
The finding that fewer than 2% had randomized trial support is striking, but it should not lead to the lazy conclusion that every non-randomized imaging AI study is worthless.[2] Some tools are narrow workflow aids. Some triage products aim to change notification timing rather than final diagnosis. Some quality-control tools reduce manual steps without claiming to improve patient outcomes directly. For those uses, a well-described prospective validation, a silent trial in the local environment, or a carefully measured pre-post workflow study may be more practical than a randomized trial.
The standard should match the claim. A device marketed as a background prioritization aid needs evidence that it detects the target condition reliably enough in the relevant imaging stream and that its alerts can be handled without harming other work. A device promoted as changing clinical outcomes needs evidence closer to the patient-level consequence being claimed. A device sold on reduced turnaround time needs data that measures turnaround time in a workflow resembling the buyer’s workflow, not just algorithmic sensitivity on a curated data set.
The common failure is not the absence of one perfect evidence type. It is the use of regulatory clearance to avoid specifying which evidence type supports which claim. A receiver operating characteristic curve, a triage timestamp, a radiologist acceptance rate, and a patient outcome are different measurements. They should not be treated as interchangeable proof.
Post-Market Signals Add Context, Not Certainty
Post-market data should be read with the same discipline. Among 903 AI-enabled devices analyzed through August 2024, 43 devices, or 4.8%, were recalled after approval at an average of 1.2 years; 5.2% had adverse event reports.[3] Those figures do not prove that recalled devices are common enough to reject the category, nor do they prove that non-recalled devices are clinically effective. They show that market entry is not the endpoint of safety and performance assessment.
Predetermined change control plans are another developing piece of the puzzle. In 2025, 10% of AI/ML clearances included PCCPs.[1] These plans may become important for managing certain device modifications after authorization, but the current adoption level is still nascent. It would be premature to treat PCCPs as proof that post-market learning, monitoring, and transparency have been solved.
The Evaluation Work Starts Where the Public Summary Ends
If the public record omits study design, sample size, subgroup performance, or workflow endpoints, those omissions become the opening agenda for evaluation. A responsible hospital does not need to pretend it can reproduce the entire regulatory review. It does need a structured way to decide whether a cleared product is safe and useful for its own intended use.
A model card is a good starting artifact if it is treated as evidence infrastructure rather than marketing collateral. It should identify the cleared intended use, target condition, input requirements, training and validation populations, study design, reference standard, sample size, exclusion criteria, scanner or modality constraints, subgroup results, known limitations, and update policy. It should also state what the model is not intended to do. Ambiguity here becomes implementation risk later.
Local validation then tests the claim against the environment where the tool will live. That does not always mean a large formal trial. It may begin with a silent deployment, retrospective local testing, or a prospective shadow period in which outputs are captured but not used for patient care. The point is to compare the device’s behavior with the hospital’s case mix, scanner protocols, report conventions, alert routing, staffing patterns, and downstream review process.
A useful local protocol names the intended workflow before the device is turned on. Who receives the alert? What happens if the radiologist disagrees? Does the device change worklist order, create a separate notification, populate the report, or send a message outside PACS? Who monitors false positives, false negatives, downtime, and version changes? These questions matter because the same algorithmic output can have different consequences depending on where it enters the clinical system.
Vendor Questions That Should Be Answered Before Reliance
The vendor conversation should be specific enough that vague answers become visible. The goal is not to punish vendors for lacking randomized trials. It is to separate products with traceable, context-aware evidence from products whose best evidence cannot be evaluated by the clinicians expected to absorb the risk.
- What exact intended use was cleared, and which claims in the sales material go beyond that wording?
- What was the study design supporting clearance, and was testing retrospective, prospective, enriched, multi-site, or otherwise constrained?
- How many cases supported the reported performance, and what were the inclusion, exclusion, and reference-standard methods?
- What performance was observed by sex, age, race or ethnicity, scanner type, site, disease severity, and relevant acquisition protocol?
- What happens when the model fails, produces no result, receives out-of-distribution input, or encounters image-quality problems?
- How are updates governed, disclosed, validated, monitored, and rolled back if local performance changes?
These are not procurement formalities. They determine whether the department can design monitoring, educate users, configure alerts, and defend the deployment if performance drifts. They also determine whether the quality committee can distinguish a product that is merely cleared from a product that is ready for clinical reliance in a specific setting.
What a Local Validation Plan Should Measure
Local validation should not be reduced to a one-time accuracy check. Imaging AI changes work by entering queues, alerts, reports, communication channels, and escalation pathways. A department evaluating a triage tool, for example, may need to measure not only detection performance but also alert volume, interruption burden, agreement with radiologist interpretation, downstream imaging or consultation changes, and whether high-priority studies actually reach the right reader faster.
A hypothetical implementation makes the difference clear. Suppose a hospital evaluates a cleared chest imaging tool intended to flag a particular acute finding. The local team should not stop after confirming that the software can technically ingest images and return an output. It should define the target study types, collect a representative local sample, compare outputs against an agreed reference process, review discordant cases, measure alert routing, and decide what threshold of false positives the reading room can tolerate. If the product will reprioritize worklists, the validation should also examine what gets pushed down.
The validation period should also establish ownership. Radiologists should own clinical interpretation of outputs. Technologists should be involved when acquisition quality affects model behavior. PACS and integration teams should verify routing, latency, downtime behavior, and audit logs. Quality and safety teams should define monitoring metrics before go-live, not after a complaint. If no one owns drift monitoring, update review, and incident response, the deployment is not mature even if the device is cleared.
Clearance Is an Eligibility Signal
FDA clearance matters. It means the device can be marketed for a cleared intended use, and it gives hospitals a regulatory starting point. But for AI medical imaging, it should be treated as an eligibility signal, not as clinical proof. The evidence available to clinicians is often too incomplete to answer the questions that determine whether a tool will help, distract, or create new risk in a particular workflow.
A cleared imaging AI product should not move into clinical reliance without accessible performance evidence, local validation against the intended workflow and population, and direct vendor answers about study design, sample size, subgroup performance, monitoring, updates, and post-market evidence. That standard is not hostile to AI. It is the minimum discipline required when the consequences of a weak deployment land in the reading room.
References
- 2025 FDA AI/ML Medical Device Year in Review, Innolitics, https://innolitics.com
- Clinical Validation of Artificial Intelligence-Based Medical Devices, JAMA Network Open, 2025, https://jamanetwork.com/journals/jamanetworkopen
- Most FDA-cleared AI devices lack clinical validation data, Applied Radiology, https://appliedradiology.com
Comments
Join the discussion with an anonymous comment.