World-first safety guide for public use of AI health chatbots

As members of the public increasingly turn to AI with health concerns, University of Birmingham researchers are leading a global programme to build the first definitive guide for safely navigating health information on AI powered chatbots.

The initiative is announced today in a correspondence published in Nature Health. The project team is now inviting the public to help shape the development of The Health Chatbot Users’ Guide, a resource designed to offer a pragmatic and neutral approach that focuses on harm reduction and maximising benefits to users.

With the advent of AI Large Language Models (LLMs) such as ChatGPT, Copilot, Claude and Gemini, millions of people worldwide are already using general-purpose chatbots including to interpret symptoms and simplify medical jargon.

However, the team of academics, health professionals, and technologists warn that these tools currently exist in a governance vacuum, leaving individual users to distinguish between evidence-based insights and ’hallucinated’ or factually incorrect advice.

Dr Joseph Alderman, NIHR Clinical Lecturer at the University of Birmingham and corresponding author of the paper said: "The use of general-purpose chatbots for healthcare is no longer a hypothetical future possibility; it is a current reality. Ignoring this shift leaves the public to navigate a hazardous information landscape unaided. Our goal isn’t to discourage innovation, but to meet the public where they are. We are building this guide to ensure users have the tools and understanding they need to use these powerful tools safely."

The project team highlights several substantial risks associated with health chatbot interactions, including:

• Medical inaccuracy: AI providing plausible but incorrect medical guidance.
• The echo chamber effect: AI models optimised for agreeability may simply mirror a user’s existing (and potentially incorrect) beliefs rather than providing necessary challenge.
• Algorithmic bias: the potential for AI to reinforce social biases that exacerbate existing health inequalities.
• Data privacy: threats to the security and confidentiality of sensitive personal health information.

Dr Charlotte Blease, health AI researcher at Uppsala University and Harvard Medical School, senior researcher on the project and author of Dr Bot said:

“Health chatbots have become the world’s most accessible first opinion - often speaking to patients before any doctor does. The danger is navigating these tools without a map. Our responsibility is to ensure that first conversation informs rather than misleads, and empowers patients.”

The project is a major international effort led by researchers at the University of Birmingham and University Hospitals Birmingham NHS Foundation Trust, in collaboration with experts from over 20 institutions globally.

The guide is being co-designed and co-delivered with public partners. Three public co-investigators and a public steering group have been empowered to set the direction of the programme, ensuring the final guidance is accessible to all age groups and literacy levels.

Members of the public are encouraged to contribute their perspectives and find out more information about The Health Chatbot Users’ Guide.