Open-source (more precisely, open-weight) medical LLMs such as Google’s MedGemma or Meta’s Llama let health systems run models on their own infrastructure, inspect and fine-tune them, and keep patient data in-house. The trade-off is that support, security, validation and liability shift from the vendor to you. Performance is no longer the main reason to choose proprietary models.
Two years ago, the choice between an open model and a proprietary one in healthcare was mostly a choice between control and capability. The best models were closed, and the open ones were noticeably worse at clinical reasoning. That gap has narrowed to the point where, for many tasks, it is no longer the deciding factor.
A study published in JAMA Health Forum in 2025 by researchers at Harvard Medical School and colleagues tested Meta’s open-weight Llama 3.1 405B against GPT-4 on the difficult diagnostic cases published as clinicopathological conferences in the New England Journal of Medicine. The open model performed on par with, or better than, the proprietary one. In May 2025, Google released MedGemma, a family of open medical models built on its Gemma 3 architecture, including a 4-billion-parameter multimodal version that handles images and text and a 27-billion-parameter text model, available to developers through its Health AI Developer Foundations programme. Other major labs have since released capable open-weight models of their own.
So the question for health systems has changed. It is no longer “can open models do the job?” It is “who do we want to be responsible when they don’t?” This piece argues that open models are the right choice when control and auditability are worth more to you than someone else’s support contract, and that most organisations underestimate what taking on that responsibility costs.
What is an open-source medical LLM?
Precision matters here, because “open” is used loosely.
Most so-called open-source medical models are open-weight: the trained model parameters are published and can be downloaded, run locally and fine-tuned. The training data and full training code are often not released, and the licence may impose conditions. MedGemma, for instance, is available under Google’s Health AI Developer Foundations terms of use, and Meta’s Llama models come with their own licence rather than a standard open-source one. Some models are released under permissive licences such as Apache 2.0. Fully open models, with published data and code, exist but are rarer in medicine because clinical training data is sensitive.
Proprietary models, by contrast, are accessed through an API or a vendor’s product. You send data to the model; you never hold the weights. The vendor controls updates, pricing and availability.
For a health system, the practical differences are where the model runs, who can inspect and change it, and who carries the risk.
Are open-source medical models as good as proprietary ones?
For many tasks, close enough that other factors decide.
The JAMA Health Forum study is one signal. Benchmarks of MedGemma and similar models on medical question answering and image tasks show respectable performance for their size, particularly when fine-tuned on specialised data. And smaller open models can be tuned for narrow tasks, such as extracting data from radiology reports or classifying free-text messages, where they often match much larger general models at a fraction of the running cost.
The frontier still tends to move first in proprietary models, especially for complex multi-step reasoning, long context and multimodal tasks. If a use case needs the best available general reasoning, a leading commercial model may still be ahead for a period after each release.
But healthcare rarely deploys the frontier for its own sake. Most clinical and administrative uses are narrow, repetitive and well defined. For those, the capability gap between the best open and closed models is usually smaller than the gap between a validated and an unvalidated deployment.
Why would a health system choose an open model?
There are four serious reasons.
Data sovereignty. Running a model on your own servers, or in a cloud region you control, means patient data need not leave your environment. For health systems in Europe, the Middle East, Asia and elsewhere, where regulators and the public are wary of sending data to foreign providers, this can be decisive. It also simplifies some contractual and privacy questions in the US.
Auditability. With open weights, you can test the model exhaustively, inspect how it behaves under controlled conditions, and document exactly which version produced which output. You are not dependent on a vendor’s assurance about what changed in the latest update. For safety investigations, research and regulatory documentation, that stability is valuable.
Stability and control. Proprietary models change. Vendors update, deprecate and reprice them, sometimes with limited notice. A clinical workflow validated on one version may behave differently on the next. With an open model, you decide when to change versions and can revalidate first.
Cost at scale. For high-volume, narrow tasks, a smaller open model running on your own hardware can be cheaper than paying per token to a commercial provider, once the fixed costs are covered.
What do you give up with open models?
Mostly the things vendors quietly provide.
Support. When a commercial model misbehaves, there is a vendor to call. With an open model, your engineers are the vendor.
Security. Hosting a model means securing the infrastructure it runs on, managing access, patching dependencies and defending against attacks such as prompt injection. That is a continuing obligation.
Safety tuning. Commercial providers invest heavily in making their models refuse harmful requests and reduce fabricated answers. Open models vary. Fine-tuning can also weaken safety behaviour if not done carefully.
Regulatory responsibility. If you fine-tune an open model and deploy it for a clinical purpose, you may become the manufacturer of a medical device in regulatory terms, with the obligations that come with that under FDA rules or the EU Medical Device Regulation. Under the EU AI Act, organisations that substantially modify a model or put it into service under their own name can take on provider obligations. As we argued in our piece on device regulation as a business model, regulatory status shapes cost as much as technology does.
Liability. With a proprietary product, a contract allocates some risk to the vendor. With an open model you run yourself, the liability sits with you. That is the heart of the trade-off: open models move responsibility inward.
A worked example: a regional health service choosing a model
Take a hypothetical public regional health service in Europe that wants a language model to summarise discharge letters for community nurses.
Option A is a proprietary model from a large US provider, accessed through a cloud service with a European data residency commitment. It is highly capable, quick to set up, and comes with a support contract. But the health service’s data protection officer is uneasy about dependence on a foreign provider, the model version will change on the vendor’s schedule, and the per-use cost will rise with volume.
Option B is an open-weight model, fine-tuned on anonymised local discharge letters and hosted in the health service’s own data centre. It performs slightly below the proprietary model on the hardest letters, but well on routine ones. It needs two engineers to run and secure it, a clinical team to validate its summaries, a quality process to monitor drift, and a regulatory assessment because the fine-tuned model is now the health service’s own product.
Neither choice is obviously right. Option A buys capability and support at the cost of control. Option B buys control and auditability at the cost of responsibility. The decision should turn on which the organisation is better equipped to carry, not on a benchmark table.
The strongest counterargument: open models are a false economy for most
The strongest objection is that open models appeal to technologists and sovereignty advocates, but most hospitals cannot operate them safely. They lack machine learning engineers, security specialists and the governance capacity to act as a model developer. For them, the “free” model becomes an unsupported, poorly monitored system with no one accountable. A proprietary product, even with its dependencies, is safer because someone whose business depends on it is watching.
This is largely right for small and mid-sized providers. The answer for them is not to avoid open models entirely but to access them through intermediaries: national health IT agencies, academic medical centres or regional collaboratives that can host, validate and support shared open models on behalf of many organisations. Sovereignty concerns make that model attractive to public health systems in particular. What matters is that someone with the capacity to govern the model is accountable for it.
What this means for health leaders and policymakers
For health system CIOs and CMIOs, the decision framework is simple to state and hard to apply. Choose open when data control, auditability, version stability or high-volume cost matter more than turnkey support, and when you can fund the engineering and governance to run it. Choose proprietary when speed, frontier capability and vendor accountability matter more. Many organisations will use both.
For clinicians, the provenance of the model behind a tool matters. Ask which model is used, whether it changes, and who validated it for your setting.
For policymakers, open models are a lever for sovereignty and competition. Public investment in shared hosting, validation datasets and evaluation standards would let smaller providers benefit without each becoming a model operator. Questions of control over data, discussed in our piece on why “who owns patient data?” is the wrong question, apply with equal force to the models trained on it.
For vendors, open models compress the value of simply providing a model. Differentiation will come from integration, validation and support, which are exactly what open-model users lack.
Frequently asked questions
What is the best open-source medical LLM?
There is no single best model; it depends on the task, languages and hardware. Google’s MedGemma, released in 2025 in multimodal and text versions, is designed for medical use. General open-weight models such as Meta’s Llama family are also widely used and fine-tuned. Any choice should be validated on your own data and use case.
Is MedGemma open source?
MedGemma is an open-weight model: developers can download, run and fine-tune it, but use is governed by Google’s Health AI Developer Foundations terms rather than a standard open-source licence. Google describes it as a starting point for developers, not a finished clinical product, so it requires validation before clinical use.
Can open-source LLMs be used with patient data?
Yes, and running them on your own infrastructure can keep patient data within your control, which helps with privacy and data sovereignty. But you become responsible for security, access controls, monitoring and compliance, and a fine-tuned model used clinically may be regulated as a medical device.
Who is liable when an open-source medical AI model makes a mistake?
Generally the organisation that deploys it for a clinical purpose, and potentially whoever fine-tuned and placed it into service, rather than the original model developer. Open-weight licences typically disclaim liability. This is one of the main differences from buying a proprietary product with contractual allocations of responsibility.
Are open models cheaper than proprietary ones?
Per use, often yes, especially for high-volume narrow tasks. In total, not necessarily. Hosting, engineering, security, validation and regulatory work can outweigh savings for organisations without technical teams. Shared hosting through regional or national bodies can make open models cost-effective for smaller providers.
The open-versus-proprietary debate in medicine is no longer really about which model is smarter. It is about where you want responsibility to sit, and whether you have built the capacity to hold it.
Would your organisation run its own open medical model, or is a vendor’s accountability worth the dependence? Share your view below.
