How Strong Is the Evidence for AI in Medical Diagnostics?

The most useful examples of AI in the medical field are no longer speculative demos. They are reading CT angiograms for suspected stroke, flagging possible diabetic retinopathy, marking lesions on MRI, and queuing imaging studies for faster review. The harder question is not whether diagnostic AI exists in clinical practice. It does. The question is whether the evidence behind a particular tool is strong enough for the clinical claim being made.

That distinction matters because authorization volume and clinical proof are moving at different speeds. By the end of 2025, radiology alone had 1,104 FDA-authorized AI devices, accounting for 76% of all FDA-authorized AI devices. In a JAMA Network Open analysis of 691 devices, fewer than 2% of cleared devices were supported by randomized clinical trials, 97% used the 510(k) pathway, and the median predicate age was 2.2 years.[1] Separately, keyword-based tracking from Innolitics and The Imaging Wire counted 295 new FDA-authorized AI/ML devices in 2025, after 253 in 2024 and 221 in 2023.[2]

A 510(k) clearance is not meaningless. It can be a reasonable regulatory route when a device is substantially equivalent to a predicate and the intended use is well bounded. But it is not the same thing as showing, prospectively, that use of the software improves diagnostic accuracy, changes management appropriately, reduces time to treatment, or performs reliably across the patients a department actually sees.

So the practical reading of the field has to be specialty-specific. Some diagnostic tasks have prospective evidence or clinically meaningful validation. Others have regulatory clearance, plausible workflow value, and vendor-reported deployment stories, but limited independent proof that patient-level diagnostic decisions improve.

Clearance Answers a Different Question Than Clinical Validation

FDA authorization asks whether a device meets the requirements of its regulatory pathway. For most AI diagnostic products, that pathway is 510(k), where the central comparison is substantial equivalence to a legally marketed predicate. Clinical validation asks a different set of questions: did the tool help clinicians detect disease in real patients, under realistic workflow conditions, with an acceptable balance of misses, false positives, interruptions, and downstream work?

The gap becomes important when a cleared device is described as clinically proven without specifying the kind of proof. A retrospective reader study, a standalone algorithm performance table, a single-site deployment account, and a randomized clinical trial do not carry the same weight. They may all be useful, but they answer different questions.

Generalizability is part of that same evidence question. A 2025 Biomedicines analysis found that only 15.5% of 2024 FDA-authorized AI device summaries reported race or ethnicity data.[3] That does not prove the devices perform poorly across groups. It means many summaries leave clinicians without the information needed to judge whether a performance claim travels safely from the validation population to their own patient population.

Radiology: The Most Mature Market, Not the Most Uniform Evidence Base

Radiology is the center of diagnostic AI adoption because the specialty has the right raw material: high-volume digital images, structured workflows, archived studies, and well-defined detection tasks. Chest imaging, neuroimaging, mammography support, pulmonary embolism triage, fracture detection, lung nodule detection, and quantitative measurements are all natural targets for software assistance.

It is also the specialty where the clearance-versus-validation problem is easiest to hide. A radiology AI product may be FDA-cleared, integrated into PACS, and visible on the worklist, yet still have limited evidence that it improves final diagnostic accuracy or patient outcomes in the setting where it is being sold. The JAMA Network Open finding is uncomfortable precisely because radiology has the largest device count: 1,104 authorized radiology AI devices, but fewer than 2% of cleared devices in the analyzed set supported by randomized clinical trials.[1]

For a department, the relevant question is rarely “does the model detect a finding in a curated test set?” It is “what happens to the reading room?” A triage tool can move a suspected emergency study upward, but it can also generate false-positive interruptions. A detection overlay can help a reader notice a subtle abnormality, but it can also normalize second-guessing when the overlay is absent. A measurement tool can save time, but only if the measurement is trusted enough to prevent duplicate manual work.

Vendor-published customer stories can be useful for understanding deployment mechanics: how alerts are routed, who receives notifications, whether the tool sits inside the existing viewer, and which staff monitor exceptions. They should not be read as independent clinical validation. If a customer testimonial reports fast notification times or operational gains, those numbers belong to that deployment account unless they are replicated in independent, appropriately designed studies.

Radiology AI is therefore both mature and uneven. The field has real operational experience, many authorized products, and tasks where software can plausibly reduce delay or improve detection. But “radiology AI” is too broad a category to judge at once. A tool for large-vessel occlusion triage, a nodule detector, a fracture flag, and a breast imaging aid sit in different clinical pathways and impose different burdens when wrong.

Neurology: Stroke and Epilepsy Show What Task-Specific Evidence Can Look Like

Neurology is a useful counterweight to the device-count story because some of the strongest examples are not broad claims about AI replacing interpretation. They are narrow diagnostic tasks where timing and lesion detection matter.

In stroke imaging, AI software for brain scans was reported to be twice as accurate as professionals in a 2,000-patient trial from Imperial College London and the University of Edinburgh.[4] The clinical interest here is not abstract model superiority. Stroke imaging sits inside a time-sensitive pathway where a delayed or missed imaging interpretation can affect treatment eligibility and transfer decisions. A tool that performs well in that setting has a clearer route from diagnostic performance to clinical usefulness than a general-purpose image classifier.

Epilepsy imaging offers another concrete example. A 2025 JAMA Neurology study reported that an AI tool detected 64% of epilepsy brain lesions previously missed by radiologists, after training on MRI scans from more than 1,100 adults and children globally.[5] That finding does not mean every MRI lesion problem is solved. It does show why prospective or clinically anchored validation matters: the outcome being measured is close to a real diagnostic failure mode, namely lesions that human readers had not identified.

For neurology teams, the evaluation should stay tied to the pathway. A stroke triage system needs evidence about speed, accuracy, escalation, and false alarms. An epilepsy lesion tool needs evidence about missed lesions, reader interaction, and whether additional detections change surgical evaluation or management. These are not interchangeable forms of proof, even if both products are described as neuroimaging AI.

Ophthalmology: IDx-DR Remains a Landmark Because the Claim Was Specific

Ophthalmology has one of the clearest diagnostic AI landmarks: IDx-DR, the first autonomous AI diagnostic device authorized through the FDA De Novo pathway for diabetic retinopathy detection. In a study of about 900 patients, the device correctly identified diabetic retinopathy about 90% of the time.[6]

The importance of IDx-DR is not that it proves autonomous diagnosis is broadly ready across medicine. Its importance is that the use case was narrow enough to evaluate: a defined disease, a defined image type, a defined diagnostic threshold, and a defined intended setting. That is exactly the kind of containment clinicians should look for when a product claims autonomy or near-autonomy.

Diabetic retinopathy screening also has a workflow argument that is stronger than many AI pitches. The clinical problem is not only whether an ophthalmologist can identify disease on an image. It is whether patients who need screening can be identified, imaged, and referred appropriately without requiring every image to begin inside a specialist clinic. The AI claim is therefore tied to access and routing, not just image classification.

That does not remove the need for monitoring. Autonomous tools still need image-quality controls, referral rules, failure handling, and population performance review. But IDx-DR is a stronger example than many diagnostic AI products because the regulatory and clinical claim was built around a specific task rather than a loose promise of better diagnosis.

Cardiology, Pathology, and Dermatology: Useful Tools, Thinner Shared Conclusions

Cardiology, pathology, and dermatology are often listed alongside radiology in discussions of diagnostic AI, but the evidence cannot be treated as if it moves as one block. Each specialty has plausible and already deployed diagnostic applications. Each also has a different failure mode when software is overtrusted.

Cardiology

Cardiology AI commonly appears around ECG interpretation, rhythm detection, imaging measurements, and risk signals derived from cardiac data. The attraction is obvious: high-volume signals, time-sensitive decisions, and measurable patterns. But a cleared ECG or imaging support tool should still be judged by the clinical action it affects. Does it change who is reviewed, how quickly, or with what confirmatory testing? Does it reduce missed disease, or mainly add another alert stream?

A cardiology department does not need to reject AI because randomized evidence is scarce for a specific product. It does need to avoid turning regulatory clearance into a claim that outcomes have improved. For some products, the best initial justification may be workflow standardization or measurement support rather than a proven diagnostic outcome benefit.

Pathology

Pathology AI depends heavily on the digital pathology environment around it. A slide-analysis tool can be technically impressive and still fail operationally if scanning quality, staining variation, specimen preparation, case mix, or review workflow differs from the validation setting. The diagnostic object is not just an image; it is a tissue specimen moving through a laboratory process.

The stronger pathology claims will be those that specify specimen type, staining protocol, target finding, reader role, and confirmation process. A tool that helps prioritize or highlight suspicious regions should be evaluated differently from one positioned as making or excluding a diagnosis. The burden of proof rises as the tool moves closer to independent diagnostic authority.

Dermatology

Dermatology AI has an intuitive appeal because skin findings are visible and image-based. That appeal can also make the field vulnerable to overgeneralization. A lesion classifier used for triage, a patient-facing image tool, and a clinician decision-support system create different risks. The clinically important question is not whether the software can sort images in a test set, but whether its use changes biopsy, referral, reassurance, or follow-up decisions safely.

Dermatology also makes subgroup and acquisition details hard to ignore. Lighting, camera quality, lesion type, skin tone, and who captures the image can all affect performance. When device summaries do not report race or ethnicity data, clinicians lose one route for judging whether a validation result is likely to generalize to their own patients.[3]

How to Read a Diagnostic AI Evidence Claim

The evaluation does not need to begin with enthusiasm or suspicion. It can begin with separating four documents that are too often blended together in sales conversations: the FDA authorization, the validation study, the workflow deployment evidence, and the local monitoring plan.

• Regulatory pathway: Was the product cleared through 510(k), authorized through De Novo, or approved through another pathway, and what intended use was actually authorized?
• Clinical validation: Was performance tested prospectively, retrospectively, in a reader study, in a randomized trial, or mainly through technical validation?
• Population reporting: Did the evidence describe age, sex, race, ethnicity, scanner or acquisition context, disease prevalence, and relevant subgroups?
• Workflow effect: Who sees the AI output, when do they see it, what action is expected, and who handles false positives or failed analyses?
• Post-deployment review: Will the department measure alert burden, missed cases, override patterns, subgroup performance, and downstream testing after go-live?

This is also where many vendor case studies should be placed. They can show that a hospital managed integration, trained staff, and routed notifications in a particular way. They should not be used as substitutes for independent evidence unless the design supports that role. A fast alert in one health system may be operationally impressive without proving that diagnostic accuracy or outcomes improve elsewhere.

The most defensible position is proportionality. A low-risk measurement aid may not need the same evidence package as an autonomous diagnostic system. A triage tool used to prioritize review may be acceptable with different proof than a tool used to reassure a patient that no specialist evaluation is needed. The evidence requirement should rise with the clinical consequence of being wrong.

The Specialty Matters More Than the Label

Diagnostic AI is already part of clinical medicine, especially in imaging-heavy specialties. Radiology has the largest authorized footprint. Neurology has examples where task-specific evidence is becoming more clinically persuasive. Ophthalmology has a landmark autonomous diabetic retinopathy device with a defined claim and prospective performance data. Cardiology, pathology, and dermatology have credible use cases, but the strength of evidence has to be read at the level of the device, task, population, and workflow.

That is the discipline clinicians and health IT leaders need after the conference demo ends. FDA authorization, prospective validation, subgroup reporting, and real-world workflow evidence are separate checks, not interchangeable badges. Readers who want the broader implementation problem can continue with why promising medical AI research often fails to become dependable clinical deployment.

References

1. Generalizability of FDA-Approved AI-Enabled Medical Devices — JAMA Network Open, 2025. link
2. FDA Authorized AI/ML Medical Devices — Innolitics / The Imaging Wire, 2026. link
3. Machine Learning-Enabled Medical Devices Authorized by the FDA in 2024 — Biomedicines, 2025. link
4. AI software for stroke brain scans trial — Imperial College London / University of Edinburgh. link
5. AI tool for epilepsy brain lesion detection — JAMA Neurology, 2025. link
6. FDA permits marketing of artificial intelligence-based device to detect certain diabetes-related eye problems — FDA, 2018. link