Most health systems should buy AI rather than build it, and put their own money into governance: local validation, monitoring and the authority to switch tools off. Building makes sense only for a narrow set of problems where your data is unique, the workflow is yours alone, and you can fund maintenance for years, not just development.
Every large health system now has someone who has fine-tuned a language model over a weekend. It works on the test set. The demo goes well. A steering committee asks whether the organisation should build its own AI instead of paying a vendor, and the room divides along predictable lines: the data scientists say build, procurement says buy, and nobody asks who will be retraining the model in 2029.
This is the wrong debate. The hard part of clinical AI is not creating a model. It is keeping a model safe and useful in a hospital that changes every month: new order sets, new coding rules, new patient populations, new versions of the EHR. Building concentrates that burden on you. Buying lets you share it with a vendor, if you write the contract properly. Either way, the capability every system needs, and most lack, is governance.
Are hospitals building or buying AI today?
Mostly buying, and mostly from one place. The US Office of the National Coordinator for Health IT (now ASTP/ONC) reported that 71% of non-federal acute care hospitals used predictive AI integrated with their EHR in 2024, up from 66% in 2023. Among those users, 80% used models from their EHR developer, 52% used third-party models and 50% used models they developed themselves. Many hospitals do all three.
The more revealing numbers are the gaps. Adoption was 86% in system-affiliated hospitals and 37% in independent ones; 81% in urban hospitals and 56% in rural ones; 90% among customers of the market-leading EHR vendor and 50% among customers of all other vendors. In other words, “build vs buy” is a choice mostly available to large, well-resourced systems. Everyone else is buying what their EHR vendor ships, or nothing.
That is a clue about where the real decision lies. For most hospitals, the question is not whether to build. It is whether to accept what arrives bundled in the EHR, add third-party tools on top, and how to check whether any of it works locally.
When does building your own AI make sense?
There are real success stories. NYU Langone trained NYUTron, a language model built on more than 4 billion words of its own clinical notes, and published results in Nature in 2023 showing it could predict 30-day readmission, in-hospital mortality, length of stay and insurance denial. The model was deployed prospectively in NYU’s system. Stanford, Mayo and a handful of other academic centres have built comparable internal tools.
What these examples share is instructive. They had years of clean, centralised data, research-grade engineering teams, access to serious computing, and an institutional reason to build, including publications, grants and the ability to study the tool as research. Very few health systems can match that combination, and fewer can sustain it.
Building makes sense when three conditions hold together. The problem is specific to your organisation, such as a local operational bottleneck, a unique population or a workflow no vendor addresses. Your data gives you an advantage a vendor cannot replicate. And you can commit to maintenance, meaning monitoring, retraining and incident response, for the life of the tool. If any one is missing, buy.
What does it really cost to build AI in a hospital?
The costs people forget are the ones that arrive after the launch.
A model trained on 2023 data is quietly aging from the day it goes live. Coding changes, a new lab assay, a new triage protocol or a shift in who comes through the emergency department can all degrade performance without any error message. Someone must watch for that, investigate it and retrain. That someone is a salaried data scientist who could leave, taking undocumented knowledge with them.
Then there is integration. A model that lives in a notebook is research. A model that shows up in the clinician’s workflow at the right moment needs interface work, EHR build, testing across versions and change management with frontline staff. For many internal projects, this costs more than the modelling.
Then there is regulation. In the US, some clinical decision support software falls under FDA device rules, depending on what it does and whether clinicians can independently review the basis for its recommendations. Internal tools are not automatically exempt. In Europe, the Medical Device Regulation and the AI Act apply to in-house software in ways many hospitals have not mapped.
And there is liability. When a vendor’s model fails, there is a contract, a support team and someone else’s insurance. When your model fails, it is yours.
Why buying without governance is just as risky
The case for buying is not that vendor AI is safe. It is that vendor AI can be governed more cheaply than internal AI can be maintained. Those are different claims, and the difference matters.
The canonical cautionary tale is a bought model. In 2021, researchers at Michigan Medicine published an external validation of the Epic Sepsis Model in JAMA Internal Medicine. The widely deployed proprietary model had an area under the curve of 0.63 in their hospitals, far below what had been reported. It generated alerts on 18% of hospitalised patients while missing 67% of the patients who developed sepsis. A tool bought from the largest EHR vendor, used in hundreds of hospitals, performed poorly in a place that bothered to check.
Epic has since revised its sepsis model, and the point is not about one vendor. It is that every model is local. Performance depends on your patients, your documentation habits and your workflows, and no vendor validation can promise how it will perform in your building. Regulators have started to push on this: the ONC’s HTI-1 rule requires certified EHR developers to disclose information about how their predictive decision support tools were developed and tested, giving buyers something to evaluate. But a disclosure is only useful if someone reads it.
A worked example: three sepsis options
Consider a hypothetical 600-bed hospital deciding how to improve sepsis detection. It has three options.
Option one: switch on the sepsis model bundled with its EHR. Licence cost is effectively zero, because it is included. The real cost is a small team, perhaps a clinical informaticist, a data analyst and a nurse lead for part of their time, to run a silent-mode trial for three months, compare the model’s alerts with actual sepsis cases, adjust thresholds and then monitor it quarterly.
Option two: buy a specialised third-party sepsis tool. Licence fees add a meaningful annual cost, plus integration work. It may perform better, and the vendor may offer outcome studies. The same local validation team is still needed.
Option three: build. The hospital’s analytics group proposes a model using its own data. Development takes a year. It needs at least two full-time data scientists indefinitely, an engineer for integration, and a validation study. If the group is strong, the model may outperform the others locally. If a key person leaves in year three, it becomes an orphan.
For most hospitals, option one or two plus rigorous governance wins. The governance team is the constant in all three. Building adds cost and risk on top; it does not remove the need to govern. That is the core of the argument: governance is not the price of buying, it is the price of using AI at all.
The strongest case for building
The counterargument deserves more than a nod. Vendors optimise for the median customer, not for you. Their roadmaps follow the largest clients. Their pricing rises once you are dependent. Buying everything means your organisation never develops the expertise to question what it buys, and a system that cannot build often cannot evaluate either. In-house capability can also be a strategic asset, attracting talent and research funding.
There is also a data sovereignty argument, which is stronger outside the US. A European or Asian health system may prefer not to send patient data to a foreign vendor’s cloud, and open-source models increasingly make local building feasible.
These are serious points. The resolution is not “never build.” It is to build the capability to evaluate, integrate and govern, which is valuable whatever you buy, and to build models only where the three conditions above genuinely hold. A small internal team that can validate, tune and monitor bought tools will get most of the strategic benefit of building at a fraction of the cost.
What this means for health system leaders
For chief executives and boards, the question to ask is not “should we build our own AI?” but “who in this organisation can tell us whether an AI tool is working, and do they have the authority to switch it off?” If the answer is nobody, that is the first investment.
For CIOs and CMIOs, standardise the procurement conditions for every AI purchase: access to performance data, a right to run silent-mode validation before go-live, commitments on monitoring and model updates, and exit terms that return your data. Our earlier piece on the questions hospitals ask too late when buying digital health applies directly. So does the argument that the procurement office is the most powerful quality regulator in a health system.
For data science teams, the most valuable work is often not building new models but building the infrastructure to evaluate everyone else’s: shared validation datasets, monitoring dashboards and a standard playbook for silent trials.
For independent and rural hospitals, which cannot fund this alone, the answer is pooling: regional collaboratives, shared governance services or partnerships with academic centres that can validate tools on similar populations.
Frequently asked questions
Should hospitals build their own AI models?
Only when the problem is specific to the organisation, its data provides a real advantage, and it can fund monitoring and retraining for years. Most hospitals lack at least one of these conditions. For them, buying from an EHR or third-party vendor and investing in strong local governance is cheaper and safer.
What is AI governance in a hospital?
It is the set of people, processes and authority that decide which AI tools are used, validate them locally before go-live, monitor performance and fairness over time, and can switch a tool off. It usually involves clinical, informatics, legal and quality leaders, and it is needed whether a model is built or bought.
How do you validate a vendor AI model locally?
Run it in silent mode, where it generates predictions without showing them to clinicians, for a defined period. Then compare its outputs with actual outcomes in your patients, check performance across subgroups, set alert thresholds that balance sensitivity with alert burden, and repeat the check periodically after go-live.
Is EHR-embedded AI good enough?
Sometimes. It is cheaper and easier to integrate, which is why most US hospitals use it. But embedded does not mean validated for your population. The Epic Sepsis Model’s poor external performance in a 2021 JAMA Internal Medicine study is a reminder that bundled tools need the same local checks as any other.
What are the hidden costs of building AI in-house?
Ongoing monitoring and retraining, EHR integration, regulatory assessment, security reviews, documentation and key-person risk when the developers leave. These often exceed the initial development cost. Organisations that budget only for building a model typically end up with a tool nobody is maintaining.
Buying AI is not the cautious choice and building it is not the bold one. The bold choice, and the rare one, is paying for the unglamorous team that checks whether any of it works.
Has your organisation built an AI tool it now regrets maintaining, or bought one it wishes it had built? Share your experience in the comments.
