Can You Really Trust AI Chatbots for Health Advice?

Can You Really Trust AI Chatbots for Health Advice?

For many people, getting a quick answer from an AI chatbot feels far easier than waiting weeks for a GP appointment. But as artificial intelligence becomes a go-to source of medical guidance, serious questions are emerging about just how trustworthy that advice really is.

One Woman's Mixed Experience with ChatGPT

Abi, a Manchester resident who deals with health anxiety, has spent the past year consulting ChatGPT regularly about her health concerns. She finds the experience more reassuring than a standard internet search, which tends to surface worst-case scenarios immediately.

"It allows a kind of problem solving together," she explains. "A little bit like chatting with your doctor."

In one instance, the chatbot served her well. After describing symptoms that pointed to a urinary tract infection, ChatGPT recommended she visit a pharmacist, where she was assessed and prescribed antibiotics. For someone who struggles to judge when medical attention is necessary, Abi felt the AI helped her access care without unnecessarily burdening the NHS.

However, not every interaction went so smoothly. After a hiking fall left her with intense back pain radiating into her stomach, Abi turned to ChatGPT for guidance. The chatbot told her she may have punctured an organ and needed to go to A&E immediately. She spent three hours waiting in the emergency department, only for the pain to subside on its own. The AI, she concluded, had simply got it wrong.

England's Chief Medical Officer Raises the Alarm

Abi's experience is far from unique, and it has caught the attention of England's most senior doctor. Professor Sir Chris Whitty, the Chief Medical Officer for England, has publicly voiced concern about the standard of health information being delivered by AI tools. Speaking to the Medical Journalists Association, he warned that people are already relying on these platforms even though the answers being generated are frequently "both confident and wrong."

As AI-powered responses increasingly appear at the top of internet search results, even those who aren't actively seeking chatbot advice may find themselves on the receiving end of it.

What the Research Actually Shows

Scientists at the University of Oxford's Reasoning with Machines Laboratory have conducted one of the more revealing studies into this issue. A team of doctors constructed detailed, realistic medical scenarios spanning a wide range of severity — from minor issues manageable at home to emergencies requiring an ambulance.

When AI chatbots were given all the relevant information upfront, their diagnostic accuracy reached an impressive 95%. "They were amazing, actually, nearly perfect," said lead researcher Professor Adam Mahdi.

But the picture changed dramatically when 1,300 real participants were asked to have natural conversations with chatbots in order to receive a diagnosis. Accuracy plummeted to just 35%, meaning two out of every three people received incorrect diagnoses or inappropriate care recommendations.

The problem, Professor Mahdi explains, lies in how humans communicate. "When people talk, they share information gradually, they leave things out and they get distracted." Those gaps in communication are enough to send an AI down entirely the wrong path.

One particularly alarming example involved a scenario based on subarachnoid haemorrhage — a form of stroke caused by bleeding on the brain that constitutes a life-threatening emergency. Depending on how participants described their symptoms, ChatGPT offered wildly different responses. In some cases, it recommended nothing more urgent than bed rest — advice that could prove fatal.

Interestingly, Mahdi noted that participants who used a traditional internet search tended to land on the NHS website and were generally better informed as a result.

The Hidden Dangers of Personalised-Feeling Advice

Dr Margaret McCartney, a GP based in Glasgow, highlights a psychological dimension that makes AI health advice particularly risky. Unlike a standard web search — where users can assess the credibility of a source based on the website it comes from — interacting with a chatbot creates the illusion of a personal consultation.

"It seems as though you're getting this supportive advice that's being made 'for you,'" she says, "and that probably changes the way we interpret what we're being told."

This false sense of personalisation may lead users to place far greater trust in AI responses than is warranted.

AI Chatbots and Medical Misinformation

A separate study from The Lundquist Institute for Biomedical Innovation in California examined how prone AI chatbots are to spreading health misinformation. Researchers deliberately posed questions in ways that were likely to elicit inaccurate responses, testing the robustness of several leading platforms including Gemini, DeepSeek, Meta AI, ChatGPT and Grok.

The chatbots were assessed across five key health topics: cancer, vaccines, stem cells, nutrition, and athletic performance. The findings added further weight to growing concerns about the reliability of AI-generated health content.

Should You Use AI for Health Questions?

AI chatbots are undeniably convenient, and in some situations — like Abi's UTI — they may point users in the right direction. But the evidence suggests their limitations are significant, particularly when it comes to complex or urgent medical situations.

Until the technology improves and appropriate safeguards are in place, health experts urge caution. If you are experiencing serious or worsening symptoms, consulting a qualified medical professional remains by far the safest course of action.