To measure AI’s impact on healthcare outcomes, hospitals must track what changes for patients and operations after deployment, such as mortality, complications, time to treatment or length of stay, compared against a credible control. Accuracy metrics like AUROC show whether a model can predict. They do not show whether anyone acted on the prediction, or whether patients benefited.
Most health AI is still sold on a single number: the area under the receiver operating characteristic curve, sensitivity at a chosen threshold, or agreement with expert readers. Those numbers answer an important question: can the model tell cases from non-cases? They say nothing about the question patients and payers care about: did anything get better?
The gap between those two questions is where a great deal of healthcare AI spending disappears. An accurate model that nobody acts on, or that fires when clinicians already know, or that changes decisions without changing outcomes, is an expensive dashboard. The argument of this piece is that hospitals should stop accepting accuracy as evidence of value and start requiring outcome measurement as a condition of deployment, even when that measurement is imperfect.
Why is accuracy not enough to prove AI works?
Because a prediction only helps if it triggers an action, the action is effective, and it arrives in time to matter. Each step can fail.
A model can be accurate but late. If a deterioration alert fires after the nurse has already called the rapid response team, its accuracy is irrelevant.
A model can be accurate but redundant. If it flags patients whom clinicians have already identified, it adds noise without adding information.
A model can be accurate but ignored. If it fires too often, staff learn to dismiss it, and the true positives are lost among the false ones.
A model can be accurate but unactionable. If there is no effective intervention for the thing it predicts, or no one with the time to deliver it, the prediction has nowhere to go. We made the same point about diagnostics in a test without a treatment is just bad news.
And a model can look accurate in development but not in your hospital.
What happens when AI is judged only on accuracy?
The best-known cautionary example is the Epic Sepsis Model. In 2021, researchers at Michigan Medicine published an external validation in JAMA Internal Medicine. The model, used by hundreds of US hospitals, had an area under the curve of 0.63 at their institution, much lower than reported by the developer. It generated alerts on 18% of all hospitalised patients while missing 67% of those who developed sepsis. Its value as a clinical signal was limited, and it was consuming clinician attention at scale.
The lesson was partly about external validation: every model is local. But the deeper lesson was that widespread adoption had happened without routine measurement of what the model was doing to care. Hundreds of hospitals switched it on. Few could say whether their sepsis outcomes had changed as a result.
The broader research literature shows a similar tilt. A scoping review in The Lancet Digital Health in 2024, by Ryan Han, Eric Topol, Pranav Rajpurkar and colleagues, identified 86 randomised controlled trials of AI in clinical practice. Most, 81%, reported positive primary endpoints, but these were mainly measures of diagnostic yield or performance, such as more polyps detected. The authors flagged the predominance of single-centre trials, limited demographic reporting and inconsistent reporting of operational effects as concerns for generalisability. Detecting more is not the same as patients doing better, though it is sometimes a reasonable step towards it.
What does it look like when AI improves outcomes?
The clearest example of a predictive model tied to patient outcomes comes from Kaiser Permanente Northern California. Its Advance Alert Monitor programme used an automated model to identify adult inpatients at risk of deterioration, and was rolled out in a staggered fashion across 19 hospitals between 2016 and 2019. The results, published in the New England Journal of Medicine in 2020 by Gabriel Escobar and colleagues, found that 30-day mortality after an alert was lower in the intervention period: 15.8% versus 20.4%, an adjusted relative risk of 0.84. Patients also had lower ICU admission rates and shorter hospital stays.
Three features explain why this counts as outcome evidence, and why it is rare.
The model was paired with a defined response. Alerts went to a dedicated team of remote nurses who reviewed them and, where appropriate, triggered a structured bedside response. The intervention was the model plus a workflow, not the model alone.
The rollout was staggered, which created a comparison. Hospitals that had not yet switched on the programme served as a natural control, strengthening the evidence that the change caused the improvement.
The outcome was one that matters. Mortality, ICU admission and length of stay are not proxies.
The study was observational rather than randomised, so some uncertainty remains. But it asked the right question, and designed its deployment so it could be answered.
How should hospitals measure AI impact on outcomes?
Start with a logic chain before go-live. Write down what the model predicts, what action should follow, who takes it, how quickly, and which outcome should change as a result. If any link is missing, the model is not ready to deploy.
Measure the process, not just the endpoint. Track how many alerts fired, how many were seen, how many led to an action and how long that took. Process data explains why outcomes do or do not move, and it can show problems within weeks rather than years.
Choose outcome measures that matter and can be measured with existing data: mortality, unplanned ICU transfers, time to antibiotics, readmissions, length of stay, missed diagnoses, time to diagnosis. For administrative tools, use operational outcomes such as denial rates or time to billing.
Build a comparison. The strongest designs are randomised, but staggered rollouts across units or sites, silent-mode periods where the model runs without showing alerts, and matched comparison groups are all better than a simple before-and-after. Our piece on forecasting hospital capacity makes the same point from the operations side: the forecast is easy, the action is where value is made.
Watch for harms as well as benefits. Alert fatigue, unnecessary tests, overtreatment and effects on different patient groups should all be measured.
Keep measuring. Performance and effect can drift as practice changes. A model that helped in year one may not in year three.
A worked example: two readmission models
Imagine a hypothetical hospital that deploys two AI models in the same year, both predicting 30-day readmission.
Model A has an impressive AUROC in the vendor’s validation. It is switched on across all medical wards, and its risk scores appear in a column on the patient list. Six months later, the hospital’s readmission rate is unchanged. When the quality team investigates, it finds that few staff look at the column, there is no defined intervention for high-risk patients, and the discharge team was never told the scores existed.
Model B is slightly less accurate. But before deploying it, the hospital defines an intervention: patients above a threshold receive a pharmacist medication review and a follow-up call within 72 hours. The hospital rolls it out ward by ward over six months, measuring how many flagged patients received both actions and comparing readmission rates between wards that had and had not started. After a year, it can show a modest reduction in readmissions among flagged patients, and knows which step in the workflow is the bottleneck.
Model A was the better model. Model B was the better investment. The difference was not the algorithm. It was the measurement and the workflow built around it.
The strongest counterargument: outcome trials are too slow and too expensive
The best objection is that requiring outcome evidence for every AI tool would freeze adoption. Outcome trials take years, cost millions, and may be obsolete by the time they report, given how fast models change. Many useful tools, such as those that save clinician time, have no plausible effect on mortality and should not need to show one. And we do not demand outcome trials for every new lab test or software update.
That objection is fair, and it argues for proportionality, not for abandoning outcome measurement. Not every tool needs a randomised trial. But every tool should have a defined outcome it is supposed to change, whether clinical, operational or financial, and a measurement plan proportionate to its risk and cost. A scribe can be judged on documentation time and burnout; a sepsis model should be judged on sepsis outcomes. The failure is not the absence of trials. It is deploying tools with no answer to the question “how will we know if this is working?”
What this means for hospitals, vendors and regulators
For hospital leaders, make an outcome measurement plan a condition of procurement. Budget for it, name an owner and set a review date. Be willing to switch off tools that do not move the outcome they were bought for.
For clinicians, ask what action a model’s output is meant to trigger and whether you have the time and authority to take it. An alert without an action is noise.
For vendors, accuracy data gets you through the door. Outcome data, even from staggered deployments at a few customer sites, is what will win renewals as buyers become more demanding.
For regulators and payers, market authorisation based on accuracy is necessary but not sufficient. Post-market expectations for real-world performance and outcome data, and payment tied to demonstrated benefit where appropriate, would align incentives with patients. As we argued in AI diagnostics: the evidence is strongest where the question is narrowest, the evidence base is uneven, and outcome data is the thinnest layer.
Frequently asked questions
What is the difference between AI accuracy and clinical impact?
Accuracy measures how well a model’s predictions match reality, using metrics such as AUROC, sensitivity and specificity. Clinical impact measures whether using the model changes patient outcomes, such as mortality, complications or time to treatment. A model can be highly accurate yet have no impact if its outputs are not acted on effectively.
What outcomes should hospitals track for clinical AI?
Choose outcomes the tool is designed to change and that can be measured with existing data, such as mortality, unplanned ICU transfers, time to treatment, readmissions, length of stay or diagnostic delays. Also track process measures, such as alert response rates, and potential harms like alert fatigue, overtesting and differences across patient groups.
Is there evidence that AI improves patient outcomes?
Some. Kaiser Permanente’s deterioration programme, published in NEJM in 2020, was associated with lower 30-day mortality after alerts. A 2024 Lancet Digital Health review found 86 randomised trials of clinical AI, most positive, but mainly on diagnostic performance rather than hard outcomes, and largely single-centre. Outcome evidence remains thin.
How can a hospital evaluate AI without a randomised trial?
Use staggered rollouts across wards or sites, silent-mode periods where the model runs without alerting clinicians, and matched comparison groups. Define outcomes and process measures before go-live. These designs are weaker than randomised trials but much stronger than a simple before-and-after comparison.
What is AI alert fatigue?
Alert fatigue happens when a model generates so many alerts, many of them false positives, that clinicians begin ignoring or overriding them, including correct ones. It can erase the value of an accurate model. Measuring alert volume, response rates and positive predictive value in practice helps detect and prevent it.
An AI model that is accurate but changes nothing is not a clinical tool. It is a report nobody reads, with a subscription fee attached. Measure what changes for patients, or do not be surprised when nothing does.
Which AI tool in your hospital could show an outcome change today, and which could only show an accuracy score? We’d like to know.
