The cost of deploying large language models in healthcare is mostly not the model. Token prices have fallen sharply, but hospitals also pay for EHR integration, security and privacy review, clinical validation, monitoring, and the staff time needed to check outputs. Over a tool’s life, those costs often exceed the licence or inference bill.
Ask a health system what its generative AI costs, and the answer usually arrives as a single number: a per-clinician monthly fee for an ambient scribe, or an annual cloud bill for model calls. The Peterson Health Technology Institute’s 2025 review of ambient scribes, for example, cited prices of roughly $100 to $600 per clinician per month. Those numbers are real. They are also the most visible and, increasingly, the smallest part of what it costs to run a language model safely in a hospital.
The thesis of this piece is that healthcare organisations are budgeting for LLMs as if they were software licences, when they behave more like clinical services. The expensive parts are getting the model into the workflow, proving it is safe for a specific use, and watching it after go-live. Meanwhile, the one cost everyone focuses on, the price of the model itself, is falling faster than almost anything else in technology.
How much does it cost to run an LLM in a hospital?
It helps to separate four layers.
The first is model access: the per-token price of calling a commercial model through an API, or the hardware and energy cost of running an open model on your own servers. This is what most cost discussions measure.
The second is integration: connecting the model to the EHR, the data sources it needs, the identity and access controls, and the screens where clinicians will see its output. Every use case needs its own prompts, data retrieval and user interface.
The third is assurance: validation before go-live, safety testing for the specific task, bias assessment, privacy and security review, legal review of the contract and data flows, and governance committee time.
The fourth is operation: monitoring outputs for drift and errors, handling incidents, updating prompts when the underlying model changes, retraining staff, and the human review time the workflow requires.
Vendors typically price the first layer, sometimes with part of the second. Health systems pay for all four.
Are LLM inference costs falling?
Dramatically, though unevenly. Epoch AI, which tracks the economics of AI, analysed the price of reaching fixed performance levels on benchmarks and found that it fell by between 9 and 900 times per year depending on the task, with the fastest declines in recent years. Capabilities that were premium in 2023 are commodity-priced now.
For health systems, this has two implications. First, a business case built on today’s token prices will look conservative in two years; the same workload will cost less. Second, and more important, the falling price of the model makes the fixed costs of integration, assurance and operation a larger share of the total. Those costs do not fall by an order of magnitude a year. They are mostly people.
Engineering choices still matter at scale. A 2024 study in npj Digital Medicine by researchers at Mount Sinai tested ten language models in more than 300,000 experiments on real clinical notes and found that grouping multiple tasks into a single call could cut API costs up to 17-fold, although performance degraded when too many tasks were packed together. A separate 2025 npj Digital Medicine analysis of generative AI costs in a revenue cycle use case, by researchers from the University of Michigan, Weill Cornell Medicine and the University of Minnesota, found that smaller, task-specific models could match or beat large general-purpose models on medical text classification at lower cost, and suggested a mix of local and commercial models for large systems.
The pattern is consistent: the price of intelligence is dropping, and the savings go to organisations that design their systems well. But even a well-designed pipeline must pay for the layers around the model.
Why do integration and validation cost so much?
Because healthcare has requirements that consumer AI does not.
Integration is hard because clinical data is fragmented, the EHR controls the workflow, and every output needs to land in the right place for the right person at the right time. A language model that drafts a discharge summary is useless if the clinician has to copy and paste it. Building that plumbing, testing it across EHR upgrades and maintaining it is skilled, ongoing work.
Validation is hard because LLM outputs are open-ended. A risk score can be checked against outcomes. A generated note, message or summary must be checked for accuracy, omissions, fabricated details and tone, often by clinicians reading samples. That is expensive clinician time. It must also be repeated when the vendor updates the underlying model, which can change behaviour without warning.
Privacy and security add their own costs. Sending patient data to an external model requires contractual protections, data processing agreements and technical controls. Hosting models locally avoids some of that but brings hardware, engineering and cybersecurity obligations. Every new vendor connection is also a new attack surface.
And there is the review time built into the workflow. If a physician must read and edit every AI-drafted message, the cost of that review is part of the cost of the tool, and it can erase the time the tool was meant to save.
A worked example: pricing an AI inbox assistant
Consider a hypothetical health system with 1,500 clinicians that wants an LLM to draft replies to patient portal messages. The numbers are illustrative, chosen to show proportions rather than to price any real product.
Inference: suppose the system handles 3 million messages a year, and each draft consumes a few thousand tokens including retrieved chart context. At current commercial prices for a capable model, the annual inference bill might be in the tens of thousands of dollars, perhaps less with task batching and a smaller model for routine messages.
Integration: building retrieval from the chart, prompt design for different message types, the drafting interface and testing across EHR versions might take a small team several months, then ongoing maintenance. That is likely several hundred thousand dollars in the first year.
Assurance: a validation phase in which clinicians review hundreds of drafts across specialties, a bias and safety review, privacy and legal work, and governance committee time. Another significant six-figure sum, and a recurring one each time the model changes.
Operation: monitoring for errors, an incident process, periodic re-review of samples and support staff. A continuing cost that scales with use.
In this sketch, the model itself accounts for a small fraction of the first-year total. Even if the inference price fell to zero, most of the cost would remain. That is why “the model is getting cheaper” does not mean “AI is getting cheap.”
The strongest counterargument: platforms absorb the hidden costs
The best objection is that most hospitals will never face these costs directly. They will buy LLM features bundled into their EHR or from vendors who do the integration and validation once and spread it across hundreds of customers. The vendor pays for the plumbing; the hospital pays a subscription. Per hospital, the hidden costs shrink.
This is partly right, and it is a strong argument for buying rather than building, which we made in our piece on AI procurement. But the hospital cannot outsource all of it. Local validation is still needed because performance depends on local patients, documentation habits and workflows. Governance, monitoring and clinician review time stay in-house. And vendor subscription prices are not tied to falling inference costs: they reflect the value the vendor believes it delivers, and the costs it has absorbed. The hospital pays for integration either way, directly or through the price.
What should health systems budget for?
A useful rule is to treat the licence or inference cost as the starting point, not the total, and to budget explicitly for each other layer. Ask for a total cost of ownership over three to five years that includes integration, validation, governance, monitoring and review time.
Design for cost efficiency. Use the smallest model that meets the accuracy requirement, batch tasks where it is safe, cache repeated context and route simple work to cheaper models. The npj Digital Medicine studies suggest that these choices can cut inference costs substantially.
Negotiate contracts that share the benefit of falling model prices, and that require notice and revalidation support when the vendor changes the underlying model.
Measure the review burden. If clinicians spend as much time checking outputs as they would have spent writing them, the tool is not saving money, whatever its price. Our earlier argument that ambient scribes reduce burnout only if nobody spends the time they save has a mirror image here: they only save money if the review time is lower than the time they replace.
What this means for CFOs, CIOs and vendors
For CFOs, stop comparing AI vendors on licence price alone. Two tools with the same subscription can have very different total costs depending on integration effort and review burden.
For CIOs, build shared infrastructure: a secure model gateway, logging, evaluation tooling and standard integration patterns. The fixed cost is spread across every use case.
For clinical leaders, own the validation and review design. Clinician time is the most expensive input in the system and the one most often left out of the business case.
For vendors, transparency about total cost of ownership will increasingly separate winners from losers, as buyers get better at spotting costs that were left off the quote.
Frequently asked questions
How much does it cost to deploy an LLM in a hospital?
It depends heavily on the use case, but inference or licence fees are usually only part of the total. Integration with the EHR, privacy and security review, clinical validation, governance, monitoring and clinician review time often add up to more than the model cost, especially in the first year.
Are open-source LLMs cheaper for healthcare?
They can reduce per-token costs and keep data on-premises, but they shift costs to hardware, engineering, security and support. For organisations without strong technical teams, the total cost may be higher. Open models make most sense where data control, auditability or high volume justify the operational burden.
Why is validating an LLM for clinical use expensive?
Because outputs are free text, they must be checked for accuracy, omissions and fabricated content, often by clinicians reviewing samples across specialties. Validation must also be repeated when the underlying model changes. That clinician time is costly, and it is frequently left out of AI business cases.
Will falling AI prices make healthcare AI cheap?
Model prices are falling quickly, but most deployment costs are people: integration engineers, informaticists, clinicians reviewing outputs and governance staff. Those costs do not fall at the same rate. Cheaper inference makes more use cases viable, but it does not remove the fixed costs of deploying AI safely.
How can hospitals reduce LLM costs?
Use the smallest model that meets accuracy needs, batch compatible tasks, cache repeated context, route simple requests to cheaper models, build shared integration and monitoring infrastructure across use cases, and negotiate contracts that pass on falling model prices and require revalidation support when models change.
The model is becoming the cheapest thing in the room. The people who make it safe, useful and trusted are not. Budget for them first.
What is the biggest hidden cost your organisation discovered after deploying a language model? Tell us in the comments.
