AI model drift in healthcare is the gradual loss of a model’s accuracy after deployment. It happens because the patients, practices, equipment or data it sees no longer match the data it learned from. A clinical algorithm validated at launch can become miscalibrated or unsafe without any change to its code, which is why it needs monitoring after go-live.
In April 2020, Michigan Medicine switched off a sepsis alert. The model, a widely used proprietary tool built into its electronic health record, had not been hacked or updated. It simply started firing too often. COVID-19 had changed who was in the hospital and what their fevers meant, and the relationship between the model’s inputs and bacterial sepsis no longer held. The hospital’s clinical AI governance committee decommissioned it. Samuel Finlayson and colleagues described the episode in a 2021 letter to the New England Journal of Medicine as a textbook case of dataset shift.
The pandemic made that shift sudden and obvious. Most drift is neither. It arrives as a slow change in coding habits, a new lab analyzer, a new treatment guideline or a gradual change in the local population. The model keeps producing numbers with the same confident precision. They are just a little more wrong each month.
My argument is that drift is not an exotic failure. It is the default path of any clinical model left alone, and the uncomfortable part is that the model’s own success can cause it. Hospitals treat go-live as the end of validation. It should be the start of surveillance.
What is model drift in healthcare AI?
“Drift” is a loose umbrella for several distinct problems. They need different fixes, so it helps to name them.
Data drift, also called covariate shift, is when the inputs change. A hospital opens a new oncology wing and the mix of patients changes. A lab switches troponin assays and the numbers mean something slightly different. The EHR vendor changes how a field is recorded. The model sees values it rarely saw in training.
Concept drift is when the relationship between inputs and outcome changes. A new treatment makes a once-dangerous finding less dangerous. A new definition of sepsis changes what counts as the outcome. The inputs look the same, but they predict something different.
Label and workflow drift is when the way outcomes are recorded changes. When the United States moved from ICD-9 to ICD-10 coding in October 2015, any model that used diagnosis codes as inputs or outcomes met a different vocabulary overnight.
Feedback drift is the subtle one. When clinicians act on a model’s predictions, they change the outcomes the model was built to predict. A deterioration model that prompts early intervention prevents some of the deteriorations it would have predicted. Retrain it naively on the new data and it may learn that its own warning signs are less dangerous than they are.
Performance can also split. A model’s overall accuracy can hold steady while it degrades for one group, such as patients whose language, age or insurance status changed in the hospital’s population. As we argued in bias in clinical algorithms, subgroup checks are the ones hospitals skip most often.
How often do clinical AI models fail after deployment?
There is no reliable national count, which is part of the problem. But there is enough evidence to show that the gap between vendor performance and local reality is common.
The best-known example is an external validation of the same proprietary sepsis model, published in JAMA Internal Medicine in 2021 by Andrew Wong and colleagues at Michigan, using data from before the pandemic. Across more than 38,000 hospitalizations, the model achieved an area under the curve of 0.63, far below the vendor’s reported figures. It missed about two thirds of sepsis cases and generated alerts for 18 percent of all hospitalized patients.
That study was about a model that underperformed from the start in a new setting, which is a local validation failure. Drift is its slow cousin, and it has been documented for years. In 2017, Sharon Davis and colleagues showed in the Journal of the American Medical Informatics Association that acute kidney injury models trained on Veterans Affairs data lost calibration over the following years, with some model types drifting faster than others. Calibration, the match between predicted risk and actual risk, often degrades before discrimination does. A model can still rank patients correctly while telling you a 10 percent risk when the true figure is 20.
Do hospitals monitor AI models after go-live?
Some do. Many do not do it thoroughly, and national data only tell part of the story.
An analysis by Paige Nong and colleagues in Health Affairs, published in January 2025 and based on the American Hospital Association’s 2023 IT survey, found that 65 percent of US hospitals used predictive models integrated into their EHR. Of those, 61 percent said they evaluated the models locally for accuracy, and only 44 percent for bias. A later federal data brief from ASTP/ONC, published in September 2025, reported that 71 percent of hospitals used predictive AI in 2024, and that among them, about 79 percent reported some post-implementation monitoring.
Those numbers are self-reported and say nothing about depth. “Monitoring” can mean a monthly dashboard reviewed by a data scientist with authority to pull a model, or it can mean someone noticing complaints. The same brief found that 80 percent of hospitals using predictive AI relied on models from their EHR developer, which often arrive switched on by default, with thresholds set elsewhere.
A worked example: the deterioration model that got too good
Here is a hypothetical, built from mechanisms that are well described in the literature.
A 500-bed hospital deploys an early warning model that predicts which ward patients will need ICU transfer within 24 hours. Validation on local historical data looks strong. The hospital sets an alert threshold and routes alerts to a rapid response nurse.
Year one goes well. The rapid response team sees patients earlier and ICU transfers from the wards fall.
Year two, three things change. The hospital replaces its lactate analyzer, which reads slightly lower. A new sepsis bundle means antibiotics start sooner. And because the alert prevents many deteriorations, patients who would have been labeled “transferred to ICU” are now labeled “stayed on ward.”
Nothing is wrong with the software. But calibration has slipped: the model now overestimates risk for some patients and underestimates it for others with the new lactate values. Alert volume rises. Nurses start dismissing alerts. Someone suggests retraining on the last year’s data. That would teach the model that the early warning signs are less dangerous than they are, because the team intervened on them.
A hospital with a monitoring plan catches this in month 14, not month 30. It tracks alert volume, positive predictive value and calibration by quarter. It logs changes to assays, order sets and workflows as “events that may affect models.” It knows that outcomes after intervention are not clean labels. And a named person has the authority to adjust the threshold or pause the model while the vendor investigates.
What is the strongest argument against continuous AI monitoring?
The strongest argument is cost and competence. Proper monitoring needs data engineers, clinical informaticists and statisticians. Many community hospitals have none. Monitoring every model continuously, with subgroup analysis, may cost more than the model saves. Some would add that it is the vendor’s job, and that hospitals should not have to build a surveillance program for a product they bought.
A related point: retraining is risky too. Every update is a new model, with new ways to fail. A stable, somewhat miscalibrated model that clinicians understand may be safer than a frequently retrained one they do not.
Both points have merit. Monitoring should be proportionate. A model that suggests appointment slots does not need the same scrutiny as one that triggers antibiotics. And retraining should be deliberate, not automatic. But the vendor argument breaks down on a basic fact: only the hospital sees its own data changing. The vendor does not know you switched lactate analyzers. Drift is local, so at least some of the watching has to be local too. Vendors can and should supply the tools. Hospitals have to look.
What this means for hospitals, vendors and regulators
For hospitals, write a monitoring plan before go-live, not after. For each model, define what you will track (alert volume, positive predictive value, sensitivity against a sampled outcome review, calibration and subgroup performance), how often and who reviews it. Keep a log of operational changes, such as new assays, order sets, coding rules and patient populations, and review models when those change. Give someone the authority to pause a model. Our piece on predicting hospital capacity makes the related point that a forecast is only as useful as the process built to act on it.
For vendors, ship monitoring with the model. Provide dashboards, local performance reports and clear guidance on which operational changes are likely to affect performance. Tell customers when your own training data or thresholds change.
For regulators, drift is a postmarket problem, and postmarket surveillance for AI remains thin. For FDA-regulated devices, predetermined change control plans govern planned updates, but they do not detect unplanned decay. Most predictive models built into EHRs are not regulated devices at all. Requirements for real-world performance monitoring, and a simple way for hospitals to report degradation, would help more than new premarket rules. As we argued in AI diagnostics, evidence that holds in one setting does not travel automatically.
Frequently asked questions
What is the difference between data drift and concept drift?
Data drift means the model’s inputs change, such as a new patient mix, a new lab assay or a change in how a field is documented. Concept drift means the relationship between inputs and the outcome changes, for example because a new treatment makes a warning sign less dangerous. Both can reduce accuracy, but they need different fixes, from recalibration to retraining with new outcome definitions.
How can hospitals detect AI model drift?
Track a few measures on a regular schedule: alert or prediction volume, the share of alerts that turn out to be true, sensitivity against a sample of confirmed outcomes, and calibration, meaning whether predicted risks match observed rates. Break these down by key patient groups. Also review models whenever operations change, such as new equipment, order sets or coding rules.
How often should clinical AI models be revalidated?
There is no universal interval. High-impact models that drive treatment deserve at least quarterly performance review, plus review after any significant change in equipment, workflow, coding or population. Lower-risk models can be checked less often. The key is a written schedule, named reviewers and a threshold that triggers action, rather than waiting for complaints.
Is it safe to retrain a clinical AI model on recent hospital data?
Not automatically. If clinicians acted on the model’s predictions, recent outcomes are shaped by those actions, and naive retraining can teach the model that real warning signs are harmless. Retraining also creates a new model that needs its own validation. It should be planned, tested on held-out data and, for regulated devices, done within the manufacturer’s authorized process.
Why do AI models perform worse after the COVID-19 pandemic?
The pandemic changed patient populations, admission patterns, testing practices and the meaning of common signs such as fever. Models trained on pre-2020 data met a different hospital. Some, like the sepsis alert Michigan Medicine switched off in April 2020, failed quickly. Others drifted more slowly, which is why any model trained on older data should be checked against recent performance.
The takeaway
A validated clinical algorithm is a snapshot of a hospital that no longer exists. Every new assay, guideline and patient changes the picture, and a successful model changes it fastest because clinicians act on it. Hospitals that treat go-live as the finish line will keep discovering drift through tired nurses ignoring alerts. The ones that treat it as the start of monitoring will catch it while it is still a number on a dashboard.
Has your organization ever switched off an AI model because it stopped working? What tipped you off?
