Asking ChatGPT or another AI chatbot about symptoms can help you understand terms, prepare questions and think through options, but it is not a reliable way to diagnose yourself or decide how urgently you need care. In the best controlled test so far, ordinary people using chatbots chose the right next step no better than people using a search engine. Treat it as a research assistant, never as a clinician.
A 60-year-old man read about the harms of table salt and asked a chatbot what he could replace chloride with. He came away believing sodium bromide was a reasonable swap, bought some online and used it for three months. He arrived at an emergency department convinced his neighbour was poisoning him. The doctors who treated him, writing in Annals of Internal Medicine: Clinical Cases in August 2025, diagnosed bromism, a form of poisoning that was common a century ago and is almost unheard of now. He spent three weeks in hospital.
It is an extreme case, and the authors never saw his original conversation. But it captures the problem better than any benchmark. The chatbot did not need to be wrong about chemistry to cause harm. It only needed to answer a question the man had framed badly, without asking him why he wanted to know.
That is the thesis of this piece. The accuracy of AI on medical questions is no longer the main risk. The main risk is the conversation: what you tell it, what it fails to ask, and how you read what comes back.
How many people are asking chatbots about their health?
Plenty, and the number is climbing fast. A KFF poll published in March 2026 found that about a third of US adults (32%) had turned to an AI chatbot for health information in the past year: 29% for physical health and 16% for mental health.
The reasons are revealing. Most users wanted quick answers. But 19% said they could not afford to see a health professional, and 18% said they had no regular doctor or could not get an appointment. For a meaningful share of people, the chatbot is not a supplement to care. It is the care they can reach.
Two more numbers from that poll should worry anyone who works in health. Among people who asked a chatbot about physical health, 42% did not follow up with a doctor. And 41% of AI users said they had uploaded personal medical information, even though 77% of the public said they were concerned about the privacy of medical data shared with AI.
How accurate is ChatGPT medical advice?
It depends on who is holding the phone, and that is the finding that matters most.
On exam-style questions, large language models now score very well. They pass licensing exams and handle textbook vignettes with ease. For a while that led to a simple assumption: if the model knows the medicine, a patient using it will get good medicine.
A team at the University of Oxford tested that assumption directly. Their randomized, preregistered study, led by Andrew Bean and published in Nature Medicine in February 2026, gave 1,298 UK participants realistic medical scenarios written by doctors. Some used one of three leading language models; a control group used whatever they would normally use at home, usually a search engine.
When the researchers fed the scenarios straight into the models, the models identified the relevant condition in 94.9% of cases. When members of the public used those same models, they identified a relevant condition in fewer than 34.5% of cases and chose the right course of action, such as seeing a GP or going to A&E, in fewer than 44.2%. Neither result was better than the control group.
The researchers traced the gap to the conversation. People left out details the model needed. Small changes in wording changed the answers. And the replies often mixed sound advice with poor advice, leaving participants to guess which part to act on.
This is a different picture from an older, widely shared finding. In a 2023 study in JAMA Internal Medicine, Ayers and colleagues had licensed clinicians compare chatbot answers with physician answers to questions posted on a public online forum. The evaluators preferred the chatbot’s responses in 78.6% of their ratings and scored them higher for both quality and empathy. Both studies can be true at once. A chatbot can write a better reply to a well-formed question than a rushed doctor typing on a forum, and still leave a worried person at home with the wrong plan.
Why does a smart chatbot give worse answers to real people?
Think about what a good clinician does in the first two minutes of an appointment. They ask when it started. They ask what makes it worse. They notice you are pale, or breathless, or guarding your side. They ask about medicines you forgot to mention. Much of diagnosis is knowing which question to ask next.
A chatbot answers the question you typed. Most will ask a follow-up if pushed, but they are built to be helpful, and helpful usually means producing an answer now. You, meanwhile, are the least reliable narrator of your own symptoms. You tend to lead with what frightens you, skip what seems irrelevant and describe pain in words that mean different things to different people.
Put those together and you get the Oxford result: excellent medical knowledge, poor outcomes. The model is not the weak link. The interface between a scared human and a fluent machine is.
There is a second, quieter issue. Chatbots are persuasive. They write in calm, organised paragraphs that sound like expertise. A search engine shows ten sources that disagree; a chatbot shows one confident voice, which makes it easy to stop looking.
A worked example: two ways to ask about chest pain
This example is hypothetical, but it is built on the failure modes the Oxford team described.
Priya, 52, has had a tight feeling in her chest on and off for two days. She types: “What causes chest tightness?” The chatbot returns a thoughtful list: muscle strain, acid reflux, anxiety, asthma, and heart problems, with a note to seek care if it is severe. Priya decides it is probably reflux because she had a spicy dinner. The list was accurate. The conclusion was hers, and it was built on the one detail she happened to find reassuring.
Now imagine she asks differently. “I’m 52, I have high blood pressure and my father had a heart attack at 58. For two days I’ve had chest tightness that comes on when I climb stairs and eases when I rest. Should I be seen today?” A good model given that description is far more likely to tell her that exertional chest discomfort with her risk factors needs prompt medical assessment, today.
Same patient, same model, very different advice. The difference is information she had all along. That is why the practical guidance below is mostly about how to ask, and when not to ask at all.
When should you not use an AI chatbot for symptoms?
Some situations are simply not chatbot territory. If you have any of the following, contact emergency services or seek urgent in-person care rather than typing:
- Chest pain or pressure, especially with breathlessness, sweating or pain spreading to the arm or jaw
- Sudden weakness, facial drooping, slurred speech or confusion
- Severe difficulty breathing, or lips turning blue
- Heavy bleeding, a serious head injury or a suspected overdose or poisoning
- Thoughts of harming yourself
Beyond emergencies, be especially cautious about using a chatbot to decide on medicines, doses or supplements, about questions involving pregnancy, babies or young children, and about anything where the answer would lead you to stop a treatment your clinician prescribed. The bromide case shows how a reasonable-sounding substitution can turn dangerous.
How to use a chatbot for health questions more safely
If you do use one, these habits reduce the risk.
Use it to understand, not to decide. Ask it to explain a term your doctor used, what a condition generally involves, or what questions to bring to an appointment. These are tasks where fluency helps and a wrong detail is unlikely to hurt you.
Give context, then ask it to ask you questions. Include your age, relevant conditions, medicines and how the symptom has changed over time. Then ask: “What else would a doctor need to know?” This partly recreates the questioning a clinician would do.
Ask for the urgent reasons first. “What would make this an emergency?” is a more useful question than “What is this?” It points you at the decision that matters.
Check anything you plan to act on. Before you change a medicine, a dose or your diet in a significant way, confirm with a pharmacist, nurse line or doctor.
Be careful what you upload. Consumer chatbots are generally not covered by health privacy law in the way your hospital is. Read the privacy settings, and think twice before sharing full records with identifying details. We have written before about how health apps often sit outside HIPAA, and the same logic applies here.
The strongest case for asking the chatbot anyway
The best argument in favour of chatbots is not that they are accurate. It is that the alternative for many people is nothing.
If you cannot get an appointment for three weeks, or cannot afford one, a chatbot that explains your blood pressure readings at midnight is better than silence or a random forum. The KFF numbers show that cost and access, not curiosity alone, are driving part of this use.
This argument is strong and deserves respect. But it cuts both ways. The people most likely to rely on a chatbot instead of a clinician are also the people with the fewest alternatives when it gets something wrong. A tool used as a bridge to care is valuable. A tool used as a replacement for care, by people with no route to check it, is where the Oxford findings bite hardest.
What this means for patients, clinicians and developers
For patients, the message is simple. Use AI to become a better-prepared patient, not to skip being one. Bring what you learned to your appointment and say where it came from.
For clinicians, the patient who arrives having consulted a chatbot is now normal. Asking “what have you read, and what worried you?” is more useful than dismissing it. The same tools are already drafting replies inside clinics, which raises its own questions about what happens when doctors simply agree with the AI draft.
For developers and regulators, the Oxford study is a warning about how medical AI is evaluated. Benchmarks that feed a model a clean, complete case measure the model. They do not measure what happens when a frightened person types half a story at 2am. Testing with real users should be the standard for any tool offered to the public for health questions, a point that echoes our earlier argument that AI performs best where the question is narrowest.
Frequently asked questions
Is it safe to ask ChatGPT about my symptoms?
It can be reasonably safe for understanding general information, medical terms or questions to ask your doctor. It is not safe as a way to diagnose yourself or decide whether you need urgent care. A 2026 Oxford study in Nature Medicine found people using chatbots chose the right course of action no better than people using ordinary search. For anything urgent, contact a clinician or emergency services.
Can ChatGPT replace a doctor’s appointment?
No. Chatbots cannot examine you, order tests, see how you look or take responsibility for your care. They also answer only what you tell them, and people often leave out important details. Use a chatbot to prepare for an appointment or understand what you were told, and rely on a qualified clinician for diagnosis and treatment decisions.
Why do chatbots do well on medical exams but poorly with patients?
Exams give the model a complete, well-written case. Real people describe symptoms partially, emphasise what worries them and may not know which details matter. In the Oxford study, models alone identified the right condition in about 95% of scenarios, but people using those models did so in under 35%, because of gaps in the conversation rather than gaps in the model’s knowledge.
Is my health information private if I share it with an AI chatbot?
Often not in the way people assume. Consumer chatbots are usually not covered by the health privacy laws that bind hospitals and clinics. Check the service’s privacy settings, avoid sharing identifying details where you can, and be cautious about uploading full medical records. A KFF poll found 77% of people were concerned about this.
What is the best way to ask an AI chatbot a health question?
Give relevant context such as age, existing conditions, medicines and how symptoms have changed. Ask what would make the situation urgent. Ask the chatbot what else a doctor would need to know. Then treat the answer as background reading and confirm anything you plan to act on with a pharmacist, nurse or doctor.
The chatbot knows a great deal of medicine. What it does not know is you, and it rarely finds out unless you tell it. Use it to ask better questions of a human, not to avoid asking one.
Have you used an AI chatbot for a health question, and did it send you towards care or away from it? Tell us in the comments.
