How NLP Models Are Transforming Overdose Death Investigation

The frustrating part of overdose surveillance is not that the death investigation is casual or slow by design. A person dies, an investigation begins, toxicology is ordered, and the medical examiner or coroner office documents what it can support. The lag appears later, when the information that could guide a public health response has to move through coding and reporting systems that were never built to distinguish every fentanyl analog, stimulant, medication, or polysubstance pattern showing up in narrative text.

That is the real opening for AI in overdose investigation and public health response: not replacing the investigation, and not magically creating real-time mortality surveillance before toxicology exists, but shortening the time between a completed forensic record and usable, substance-specific public health data. The strongest recent evidence suggests that, for this narrow task, natural language processing is no longer the weak link.

The lag that matters is after the evidence is already written down

Traditional overdose mortality reporting depends heavily on ICD-10 coding. Those codes are useful for national comparability, but they are coarse for compound-level surveillance. They can tell a health department that a death involved a broad drug category. They are much less helpful when the operational question is whether fentanyl is appearing with cocaine in a particular county, whether a medication is being misclassified into a broad narcotic category, or whether a new combination is moving faster than quarterly reports can show.

The current reporting lag described in the overdose NLP literature is about 4 to 6 months for CDC provisional data, while toxicology completion can be roughly 3 weeks. The proposed gain is therefore not instant knowledge at the scene. It is the narrower, more plausible gain of moving reportable surveillance closer to the toxicology window once the relevant narrative and lab information are available.[1]

That distinction matters. A model cannot classify fentanyl from a report that has not yet received toxicology, and it cannot rescue a scanned form whose critical words never made it through optical character recognition. But if the narrative is present, legible, and standardized enough to process, the evidence now supports a serious deployment conversation.

BioClinicalBERT makes the model-performance question much less theoretical

Funnell and colleagues tested a fine-tuned BioClinicalBERT model for classifying drugs involved in overdose deaths from death certificate and autopsy-related text. The reported performance is unusually strong for a public health NLP task: macro F1 of at least 0.998 on internal validation and 0.966 on external validation. The model also classified 7,088 death certificate cases in 9.02 seconds, or 1.28 seconds per 1,000 cases.[1]

Those numbers deserve attention because the task is bounded. The model is not being asked to produce a broad clinical interpretation, summarize a decedent’s life history, or infer intent. It is being asked to map recurring forensic and toxicology language into substance labels. That is exactly the kind of repetitive, high-volume text problem where a fine-tuned clinical transformer can be more useful than a general AI demonstration.

The external validation result is especially important. Internal validation can look excellent when a model sees text from the same documentation environment that shaped its training data. External validation asks a harder question: does the model still work when wording, templates, and local reporting habits shift? A 0.966 macro F1 does not remove the need for local validation, but it does move the discussion from “can this possibly work?” to “what conditions have to be in place for it to work here?”[1]

The speed result is not just a technical flourish. In a local health department, a batch of several thousand cases classified in seconds changes where staff time goes. Analysts can spend less time converting narrative text into basic categories and more time reviewing exceptions, checking low-confidence outputs, validating unusual combinations, and deciding whether a signal should change outreach, naloxone distribution, clinician alerts, or drug-checking priorities.

The model choice is also a useful correction to the current habit of reaching first for the biggest general-purpose system. BioClinicalBERT is an encoder-only model with 110 million parameters. In the Funnell study, it outperformed decoder-only large language models including Llama 3 and Qwen 3 on this specific forensic classification task, while requiring substantially less compute.[1]

That does not mean encoder-only models are always superior. It means task fit still matters. A medical examiner office or state health department does not need a model that can chat fluently about toxicology. It needs a model that can classify named substances accurately, consistently, quickly, and cheaply enough to run inside a surveillance workflow.

Compound-level classification changes what public health can see

The payoff is not simply faster reporting of the same categories. The more important change is specificity. ICD-10 categories collapse distinct substances. In one example emphasized in the overdose NLP literature, buprenorphine and fentanyl can both fall under T40.4, even though they imply very different public health interpretations and interventions.[2]

Goodman-Meza and colleagues showed why that specificity matters. Their 2022 JAMA Network Open study classified 8,738 overdose deaths across 10 U.S. counties with near-perfect substance-level accuracy. The multi-label classification identified cocaine plus fentanyl as the most common polysubstance combination, a pattern that traditional ICD-10 reporting would obscure.[2]

That is not a cosmetic improvement. If fentanyl deaths are rising mostly through fentanyl alone, a county might prioritize one set of messages and services. If fentanyl is repeatedly appearing with cocaine, the audience, venues, outreach partners, and risk communication may change. A stimulant-using population that does not identify as opioid-using may not respond to the same warning language or service pathways.

The same study reported that 13% of overdose deaths involved more than one substance and identified 180 unique drug combinations in a single dataset.[2] That volume of combinations is exactly where manual abstraction and broad ICD-10 groupings struggle. The question is not whether every rare combination should trigger an alert. The question is whether the surveillance system can preserve enough detail for analysts to notice which combinations are becoming operationally important.

The Goodman-Meza and Funnell studies should not be treated as fully independent confirmations; they share UCLA authorship and related data foundations. The later work extends the earlier line of research with broader multi-label classification and external validation. Still, taken together, they show a coherent progression: compound-level classification is feasible, polysubstance patterns are public-health relevant, and transformer-based models can do the work with accuracy high enough to justify workflow testing.[1][2]

The evidence is not confined to one model family

A separate Tennessee-Vanderbilt project points in the same general direction without relying on the same model architecture. Tang and colleagues developed a random forest model using 17,342 autopsies and reported AUROC values of at least 0.95. The project was designed with handoff to the Tennessee Department of Health in view, which is the right test of relevance for this problem.[3]

This matters because a health department evaluating NLP does not have to treat BioClinicalBERT as the only possible answer. The broader finding is that structured drug classification from unstructured forensic text is a tractable surveillance task. Some agencies may choose a transformer model; others may start with a simpler machine-learning classifier if it is easier to validate, maintain, and explain in their environment.

The model architecture should follow the job. If the local need is batch classification of known drug terms from standardized narratives, a smaller fine-tuned model may be the responsible choice. If the workflow requires broader document triage, exception explanation, or interactive review, other systems may enter the discussion. But the benchmark should remain operational: does the output improve surveillance decisions without adding a new review burden that the office cannot staff?

Once accuracy is high, the bottlenecks move upstream and downstream

High model performance does not make the surveillance system automatic. It changes which problems deserve attention. For many agencies, the hardest barriers are likely to be the condition of the input documents and the handoff after classification.

OCR quality is one of those practical barriers that can look minor from a distance and dominate implementation up close. Handwritten notes, low-quality scans, inconsistent document layouts, and legacy PDF archives can all degrade the text before the model ever sees it. A classifier trained on clean electronic narratives should not be assumed to perform the same way on a scanned packet with marginal handwriting.

State and county variation creates a second problem. Medical examiner and coroner offices do not all write narratives the same way. Some use structured templates; others rely more heavily on free text. Some separate toxicology findings cleanly; others embed relevant information across multiple sections. External validation helps, but a local pilot still needs to test the actual documents that will feed the system.

Toxicology turnaround remains a hard boundary. The research-supported lag reduction is from a 4-to-6-month reporting delay to roughly the 3-week toxicology completion window, not to same-day overdose mortality intelligence.[1] For fast-moving clusters, that distinction matters. NLP can make death data more timely than traditional coded mortality files, but it will still arrive later than emergency medical services runs, syndromic surveillance, drug-checking results, or field reports.

Novel synthetic drugs create another kind of gap. Models trained on known substances may miss compounds that were absent or rare in the training data. That is not a reason to avoid NLP; it is a reason to build exception review, periodic retraining, and toxicologist feedback into the workflow. A system that classifies known substances well but silently mishandles emerging ones can give a false sense of completeness.

Subgroup performance is not a footnote

The Funnell study reported lower F2 scores for American Indian, Asian, age 14 and younger, and age 65 and older groups.[1] That finding should be read carefully. It does not erase the strong overall performance. It does mean that statewide or multi-county averages can hide the exact places where a surveillance system most needs review.

Small populations are easy to lose in aggregate metrics. If a model performs slightly worse for a group with fewer cases, the problem may not be obvious in headline F1 scores. In practice, that can affect which deaths are classified into which substance categories, which trends are noticed, and which communities appear in reports that drive funding and outreach.

A deployment plan should therefore include routine performance monitoring by age, race, ethnicity, jurisdiction, document source, and substance class where case counts allow meaningful review. The goal is not to make every subgroup estimate look stable when the denominator is too small. The goal is to prevent excellent average performance from becoming an excuse to stop looking.

What an operational workflow has to prove

For a health department, the useful question is not whether an NLP model is impressive in isolation. It is whether the classified output reaches the people who can act on it, early enough and specifically enough to change a decision.

• The medical examiner or coroner office needs a consistent way to provide machine-readable narratives and toxicology results.
• The model output needs review rules for low-confidence cases, rare substances, and unexpected combinations.
• The surveillance team needs a data structure that preserves compound-level and polysubstance information rather than collapsing it immediately back into broad categories.
• Analysts need thresholds or review meetings that define when a pattern becomes an alert, a partner notification, or a resource-allocation issue.
• The agency needs ongoing monitoring for document drift, new drug terms, subgroup performance, and changes in toxicology reporting.

Without those pieces, the model can become another fast process that produces data no one is positioned to use. With them, NLP can take work that now depends on slow manual abstraction and make it available closer to the point where public health teams still have time to respond.

The most grounded conclusion is also the most useful one: NLP can make overdose death surveillance faster and more substance-specific, especially for compound-level and polysubstance classification. The strongest current studies support that claim. Deployment will succeed only where agencies standardize inputs, connect classification to reporting workflows, and keep testing performance across populations, jurisdictions, and emerging drugs.

References

1. BioClinicalBERT benchmark, Funnell et al., Journal of Forensic Sciences, 2026.
2. Goodman-Meza et al. overdose death substance-level classification study, JAMA Network Open, 2022.
3. Tang et al. Tennessee-Vanderbilt random forest overdose autopsy model, JMIR Public Health and Surveillance, 2023.