Why post-market surveillance fails for AI medical devices

MAUDE looks mature from a distance. It has device identifiers, event types, dates, manufacturer narratives, patient problem codes, device problem codes, and a long institutional memory of medical device reporting. For many conventional devices, that structure is imperfect but recognizable: something happened, someone observed it, a report was filed, and a reviewer can try to reconstruct whether a device failed.

For post-market surveillance of AI medical devices, that same structure produces a different kind of confidence problem. A 2025 systematic assessment of MAUDE reports for 823 510(k)-cleared or De Novo AI/ML-enabled devices found 943 medical device reports from 2010 through 2023, then identified three failures that are less dramatic than a model hallucinating and more damaging for surveillance: reports are concentrated in a tiny slice of device types, key fields are often missing, and most incidents are flattened into the category “malfunction” even when the narrative points somewhere else. [1]

That is not a paperwork complaint. It is a traceability complaint. If a safety reviewer cannot tell where an event occurred, who reported it, whether a health professional was involved, what operating environment surrounded the model, or whether a subgroup was disproportionately affected, the report count may increase without the surveillance system becoming meaningfully more observant.

The Database Sees a Narrow Slice of the Market

The first warning sign is concentration. In the Babic et al. assessment, more than 98% of AI/ML device adverse event reports came from fewer than five device types. The authors contrasted that with roughly 85% concentration among non-AI devices, already a high baseline but meaningfully less extreme. [1]

That matters because MAUDE is often read as if the presence or absence of reports can say something about post-market risk across a device category. With AI/ML devices, the report distribution does not support that kind of broad inference. A reviewer can reasonably ask why certain device types generate reports. The harder question is whether other AI/ML devices are safer, less used, less recognized as reportable, less likely to be linked to harm, or simply less visible to the reporting system.

Those are not interchangeable explanations. A low report count could reflect good performance, but it could also reflect weak detection. In a clinical AI workflow, an incorrect output may be accepted by a user, overridden without escalation, caught by another system, or discovered only when outcomes are reviewed later. MAUDE was not built to distinguish those paths. It receives reports after an event becomes legible as a device problem.

For a compliance officer trying to answer whether a deployed model remains safe after clearance, this is a thin footing. The database may show that reports exist. It does not necessarily show whether the deployed population, site mix, workflow, input data, or user behavior has changed in ways that would make those reports interpretable.

Missing Fields Break the Reconstruction

The second failure is more administrative, and therefore easier to underestimate. In the same 2010–2023 MAUDE dataset, Event Location was missing in 100% of AI/ML device reports. Health Professional reporter information was missing in 73%, event date in 32%, and reporter occupation in 30%. [1]

A missing Event Location field is not just an empty box. For AI/ML devices, location can be a proxy for clinical workflow, scanner type, imaging protocol, EHR configuration, network integration, staffing pattern, patient mix, and local override practice. None of those details is guaranteed by a location field, but without it the first branch of the investigation is cut off.

The absence of reporter context is similarly consequential. A report from a radiologist, laboratory director, bedside nurse, service engineer, patient, or manufacturer analyst does not carry the same observational meaning. Each observer sees a different part of the failure chain. If the report does not reliably identify whether a health professional was involved or what occupation the reporter held, the database loses a basic clue about how the event surfaced.

Event date matters for ordinary devices too, but AI makes timing more important. A model version, software update, input distribution, clinical protocol, or connected system may change between deployment and report submission. Without a reliable event date, it becomes harder to align an incident with the actual model state and operating conditions at the time it occurred.

This is where the apparent orderliness of MAUDE becomes misleading. The report has a place in the system. It may have a manufacturer narrative. It may carry an event type. But if the fields needed to reconstruct the setting are empty, the report cannot do the job that post-market AI surveillance demands of it.

“Malfunction” Becomes a Catchall

The third failure is semantic. Babic et al. found that 90.88% of the AI/ML device reports were classified as “malfunction.” Yet the qualitative descriptions included incidents pointing to user errors, expired supplies, and analyst mistakes, rather than device malfunctions in the narrower sense. [1]

That is not a small labeling nuisance. A category controls what later reviewers believe they are counting. If a database marks nearly everything as a malfunction, the reviewer cannot easily separate a software defect from a training problem, a workflow mismatch, an expired consumable, an integration issue, or a human interpretation error. The incident has been stored, but its causal shape has been compressed.

The compression is especially awkward for AI/ML devices because many relevant failures occur at the boundary between model output and human action. A model may generate a low-confidence or misleading result. A clinician may over-trust it, ignore it, misunderstand it, or use it outside the intended workflow. A laboratory or imaging process may feed the model inputs that technically pass through the system but no longer resemble the data environment in which the model was validated. Calling all of this “malfunction” gives the system a label, not an explanation.

There is also a regulatory consequence. If incident types are collapsed, manufacturers and regulators cannot reliably compare like with like. A cluster of true software malfunctions should trigger a different review than a cluster of use errors after a workflow change. A set of analyst mistakes should not be read the same way as a model producing systematically degraded outputs for a subgroup. MAUDE’s categories do not force that separation.

The result is a database that can appear full while remaining under-specified. A count of malfunction reports may be high enough to prompt attention, but not structured enough to tell a safety team what kind of attention is needed.

The Hardest AI Signals Are Not Single Events

Even if every MAUDE report were complete, promptly filed, and semantically precise, the system would still miss important AI/ML safety signals. Concept drift, covariate shift, algorithmic stability, and subgroup performance disparities are not always visible as discrete patient-level adverse events. They are often population- and deployment-level patterns.

Concept drift is a good example. A model can remain technically operational while the clinical population, acquisition protocol, coding practice, disease prevalence, or treatment pathway changes around it. The device has not necessarily broken in the way a pump, lead, or implant might break. Its performance has become less certain because the world feeding it data has moved. For readers working through mitigation governance, ClinicalMind’s guide to model drift in clinical AI addresses the downstream problem of what to do once drift is detected.

Covariate shift creates a similar problem. The incoming data distribution changes, but the change may not announce itself through a reportable injury. It may first appear as reduced sensitivity in a particular clinical context, more frequent low-confidence outputs, an increased override rate, or disagreement between model output and specialist review. Those are surveillance signals, but they are not naturally captured by an adverse event form built around a reported incident.

Subgroup performance disparities are even more poorly served by event-based reporting. A model may perform acceptably in aggregate while underperforming for a demographic or clinical subgroup. Individual reports may not reveal the pattern, especially if the affected cases are dispersed across sites or if no single case is recognized as device-related. The safety signal lives in the denominator as much as in the numerator: who was exposed, how many cases were processed, what the subgroup distribution was, and how performance differed across those groups.

MAUDE has no practical way to ask those questions at scale. It can receive an incident report. It cannot, by itself, tell whether the model’s operating environment has shifted, whether the deployed population differs from the validation population, whether performance is stable across time, or whether a subgroup is carrying disproportionate error.

The FDA Recognizes the Gap, but the Tools Are Partial

The FDA is not blind to this problem. Its CDRH regulatory science research program includes work on methods and tools for effective postmarket monitoring of AI-enabled medical devices, including monitoring approaches for devices after deployment. [2]

That research posture matters because it acknowledges that conventional passive reporting is not enough. The agency’s interest in postmarket monitoring methods is a sign that the surveillance problem is not just manufacturer diligence or hospital reporting culture. It is also an infrastructure problem: the existing reporting architecture does not naturally collect the signals that adaptive or data-dependent software can generate.

Predetermined change control plans, or PCCPs, are another partial mechanism. FDA’s final guidance on marketing submission recommendations for PCCPs for AI-enabled device software functions was issued in August 2025, and the framework allows certain planned modifications to be described in advance within regulatory bounds. The early footprint was limited: 53 devices had PCCPs by the end of 2024. [3]

PCCPs help with a specific lifecycle problem: how a manufacturer can make anticipated changes without treating every bounded update as an entirely new regulatory event. They do not solve the broader surveillance problem. A planned update pathway can describe how change will be controlled, but it does not tell a regulator whether a model is silently degrading at a particular hospital, whether demographic performance is diverging, or whether workflow changes have altered the risk profile after clearance.

What a Parallel Surveillance Layer Would Need to Report

The answer is not to abandon MAUDE. Adverse event reporting still has a role when a device-associated event is observed and reportable. The problem is expecting that event-based system to carry the full burden of AI medical-device surveillance.

A useful parallel layer would be structured around deployment conditions and performance over time, not only around incidents. At minimum, it would need fields and reporting duties that make the following reconstructable:

• Where the model is deployed, including enough site and workflow context to interpret changes in performance.
• Which model version, configuration, input sources, and connected systems were active during a defined monitoring period.
• How many cases were processed, not merely how many adverse events were reported.
• How performance changed across time, clinical setting, and relevant demographic or clinical subgroups.
• Who reviewed exceptions, overrides, complaints, and discordant outputs, and how those reviews were resolved.

Quarterly deployment-condition reports would be one practical form. They would not need to read like adverse event narratives. Their value would be in regularized denominators and context: case volume, site additions, model version, workflow changes, input distribution changes, override patterns, and known limitations observed in use.

Mandatory demographic performance assessments would address a different gap. If a model’s safety claim depends on performance across a patient population, post-market reporting should not rely on the hope that subgroup failures become visible through individual complaints. Manufacturers and deploying institutions should be able to show whether performance remains acceptable across relevant groups, and regulators should be able to see when the denominator is too thin to support confidence.

Model-card-style labeling could make the surveillance layer easier to interpret. A label that records intended use, training and validation boundaries, known limitations, required input conditions, monitoring expectations, and update history gives hospitals and reviewers a reference point. Without that reference point, post-market reports arrive detached from the assumptions under which the model was cleared or deployed.

There will be burden questions, and they are real. Hospitals do not need another unbounded reporting exercise, and manufacturers cannot standardize every local workflow detail by force. But the present alternative is worse: a system in which the most important AI safety signals may remain outside the form entirely.

MAUDE was designed for a different device world. It can still record events, but AI medical-device safety also requires a surveillance layer that tracks performance, populations, and operating conditions over time. Without that layer, post-market surveillance will keep counting reports while missing the changes that determine whether the model remains safe in use.

References

1. A general framework for governing marketed AI/ML medical devices, npj Digital Medicine, May 2025.
2. Methods and Tools for Effective Postmarket Monitoring of Artificial Intelligence (AI)-Enabled Medical Devices, U.S. Food and Drug Administration.
3. Marketing Submission Recommendations for a Predetermined Change Control Plan for Artificial Intelligence-Enabled Device Software Functions, U.S. Food and Drug Administration, August 2025.