AI tools can translate and generate health information in many languages, but their quality drops sharply outside English and a few widely spoken languages. Studies of machine translation and chatbots show more errors in lower-resource languages. For patients who speak those languages, a fluent but wrong answer is a safety risk, so AI health content needs testing by native speakers before use.
A mother in a clinic waiting room types a question into a chatbot in her own language: her baby has had diarrhoea for two days, how much of the oral rehydration solution should she give? The answer comes back fluent, warm and grammatically perfect. It has also garbled a measurement, turning “a small cup after each loose stool” into something closer to “a small cup each day”.
She has no way of knowing. The fluency is the problem. A clumsy, obviously broken translation invites suspicion. A smooth one invites trust.
Most health AI is built, tested and evaluated in English first. Other widely spoken languages come second. The thousands of languages spoken by most of the world’s poorest people come a distant third, if at all. That is usually discussed as a fairness issue. It is more immediate than that. When AI health information is worse in your language, you are not getting a lesser product; you are getting a riskier one, and the risk lands on people who already face the most barriers to care.
How well does AI translate health information?
Machine translation is the oldest and best-studied form of this problem, and the evidence is instructive.
A 2021 study in the Journal of General Internal Medicine, led by Breena Taira, asked native speakers to evaluate Google Translate’s versions of 20 common emergency department discharge instructions in seven languages. Across 400 translated statements, the overall meaning was retained 82.5% of the time. But the average hid a steep gradient. Spanish was accurate 94% of the time and Tagalog 90%. Korean and Chinese were around 82%. Farsi fell to 67.5%, and Armenian to 55%. The authors concluded that the tool was inconsistent between languages and should not be relied on for patient instructions.
Machine translation has improved since then. A 2025 study in the Journal of Emergency Medicine assessed Google Translate on discharge instructions across 26 languages and found an overall success rate of 95%. But even there, 16 of the 26 languages scored 90% or below in at least one area of the instructions, and five, including Amharic, Oromo and Yoruba, scored below 90% overall. The authors could not easily predict which languages would do well.
Put simply: average accuracy is high and rising, and the average is the wrong number. The languages where performance is worst are often the ones with the fewest human interpreters to catch the error.
Do AI chatbots answer health questions differently in other languages?
Large language models add a new layer. They do not just translate; they generate answers. And the evidence suggests their medical answers vary by language.
A study by researchers evaluating large language models on real health questions in English, Spanish, Chinese and Hindi, titled “Better to Ask in English”, found a pronounced disparity in the quality of responses across languages, judged on correctness, consistency and verifiability. English responses were the most reliable. The title is its conclusion.
The reasons are structural. Models learn from the text available online, and the internet’s health content is dominated by English, followed by a handful of other major languages. For a language with little digital text, and even less medical text, a model has fewer examples to learn from. It may translate internally from English, carry over assumptions from English-language medicine, or simply produce plausible-sounding sentences with little grounding.
Efforts to close the gap exist. Meta’s No Language Left Behind project, described in Nature in 2024, built translation models covering 200 languages, many of them low-resource. Grassroots research communities such as Masakhane have built datasets and models for African languages. Health-specific projects are emerging. But the gap between the best-served and least-served languages remains wide, and medical content is a small, specialised slice of an already small pool of data.
Why is this a patient-safety problem and not just an equity problem?
Equity framing says: speakers of some languages get worse service. That is true, but it can make the problem sound like a matter of access, fixable by eventually adding more languages. The safety framing is sharper, for three reasons.
Errors are invisible to the patient. A person asking a question in their own language usually cannot check the answer against an English original. Unlike a clinician who spots an odd recommendation, a patient has no second channel.
Fluency hides failure. Modern systems produce smooth, natural text even when the content is wrong. The cues that once signalled machine translation, such as awkward grammar and literal phrasing, have largely disappeared. Trust goes up faster than accuracy.
Health instructions are unforgiving. A small error in a dose, a timing or a warning sign can cause harm. “Take twice a day” and “take two at a time” are one word apart in many languages.
This is the same dynamic we described in the real risk of AI-drafted patient messages: the danger is not obvious error but confident, plausible error that nobody checks. In a lower-resource language, there is often nobody positioned to check at all.
A worked example: rolling out a maternal health chatbot
This is a hypothetical example, built to illustrate what language testing involves.
A health ministry plans to launch a chatbot that answers questions from pregnant women and new mothers. The country has one official language, used in schools and government, and six widely spoken local languages. The vendor’s demo, in the official language, is impressive.
The ministry asks for a test in all seven languages. It recruits native-speaker midwives and nurses to write 100 realistic questions in each, including danger signs such as bleeding, severe headache and reduced fetal movement, and to score the answers.
The results split into three groups. In the official language, answers are accurate and safe in most cases. In two widely spoken local languages, answers are mostly accurate but miss danger-sign urgency in a worrying share of cases, sometimes advising rest when the correct advice is to go to a facility immediately. In the remaining four, answers are fluent but frequently wrong, including garbled medication instructions.
The ministry makes three decisions. It launches in the official language with a clear referral pathway for danger signs. In the two middle languages, it uses pre-written, clinician-verified answers for danger signs and common questions, with the AI only choosing which answer to show. In the four weakest languages, it delays the chatbot and funds voice messages recorded by local health workers instead.
That is not a failure of ambition. It is what responsible deployment looks like when you actually measure.
The strongest counterargument: isn’t imperfect information better than none?
A fair critic would argue that for many speakers of low-resource languages, the realistic alternative to an imperfect AI answer is not a perfect human answer. It is no answer at all, or advice from a neighbour, a market seller or social media. Interpreters are scarce. Written materials in local languages are rare. If AI health information is right, say, 80% of the time in a given language, it may still be better than the status quo. Holding it to a standard of perfection could deny people useful information and entrench English-speaking advantage.
This argument deserves respect. Denying information is also a harm, and many human sources of health information are far from accurate.
But the counterargument depends on knowing the error rate and where the errors fall. An 80% accurate tool that is wrong mainly about trivia is very different from one that is wrong about danger signs. Without testing in each language, nobody knows which one they have. And the combination of fluency and authority means that AI errors may be trusted more than a neighbour’s. The right answer is not to withhold AI from speakers of lower-resource languages but to test it in their languages, restrict it where it is unsafe and invest in making it better. Our earlier piece on bias in clinical algorithms made the same point: the fix that matters is the unglamorous one of checking performance in the groups the tool will actually serve.
What this means for ministries, health systems and developers
For ministries of health and hospitals, make language-specific testing a condition of deployment. Any AI tool that gives patients health information should be evaluated by native-speaker clinicians in every language it will be used in, with particular attention to danger signs and dosing. Do not accept an overall accuracy figure. Ask for results by language.
For health systems in multilingual high-income countries, the same applies. Machine translation of discharge instructions and patient letters is already common. Treat it as a clinical process with a known error profile, not a clerical convenience. For critical instructions, use professional translation or verified templates, and use qualified interpreters for conversations.
For developers, report performance by language, not just on average, and be explicit about which languages have been clinically validated. Invest in local-language health data built with and paid for by local experts. And design for fallbacks: when confidence is low, the tool should say so and route the person to a human, rather than producing a fluent guess.
For funders, language is infrastructure. Funding high-quality, clinically reviewed health text and speech data in underserved languages is one of the most direct ways to make AI safer for the people who most need it.
Frequently asked questions
Is Google Translate safe for medical instructions?
It depends heavily on the language and the content. Studies of emergency department discharge instructions have found high accuracy in some languages, such as Spanish, and much lower accuracy in others. Errors are hard for patients to detect. For important instructions, professional translation, pre-translated verified materials or a qualified interpreter are safer, with machine translation used only as a supplement.
Do AI chatbots work in African languages?
Performance varies widely. Major models can produce text in many African languages, but quality is usually lower than in English, especially for specialised medical content. Research groups such as Masakhane and projects such as Meta’s No Language Left Behind have improved translation for many languages, but health-specific accuracy should be tested by native-speaking clinicians before tools are used with patients.
Why are AI models worse in some languages?
AI models learn from available digital text, and most online content, especially medical content, is in English and a small number of other major languages. Languages with less digital text give models fewer examples, so their outputs are less accurate and less grounded. Models may also carry over assumptions from English-language medical sources that do not fit local contexts.
How can hospitals use AI translation safely?
Hospitals can limit AI translation to low-risk content, use pre-translated and clinician-verified templates for critical instructions, test tools by language with native-speaker clinicians, and keep qualified interpreters for clinical conversations and consent. Staff should know which languages a tool performs poorly in and should never rely on family members or unverified AI for high-stakes communication.
The takeaway
An AI that speaks every language fluently but only knows medicine well in English is not a universal tool. It is an English tool with a convincing accent. Until health AI is tested and funded language by language, the people it reaches last will be the people it puts most at risk.
What language do your patients speak that your AI tools have never been tested in?
