From clinical guidelines to patients: Can AI communicate trustworthy evidence?
Author: Dr Neda Kabiri1,2, Dr Hanieh Salehi-Pourmehr1,2, Dr Sakineh Hajebrahimi1,2
1. Research Center for Evidence-based Medicine
2. Tabriz University of Medical Sciences
As researchers working in evidence-based healthcare, we spend a lot of time producing and communicating evidence. We search for studies, assess their quality, bring findings together, develop or adapt clinical guidelines and try to make the final recommendations useful for healthcare professionals and decision-makers. With the rapid development of artificial intelligence (AI), a question I had been pondering became more interesting to me: Can AI help us make evidence understandable and useful for the people who actually need it?
We know that AI can summarise a long document in seconds. It can translate scientific language, simplify complex sentences, create different versions of the same information for different audiences and even turn text into visual content. We understand that these are powerful opportunities for evidence communication. But what is the risk? If we simplify evidence too much, do we risk changing its meaning? I wanted to explore this question by working through something familiar to me: a clinical guideline.
From a clinical guideline to a patient-friendly summary
I chose an adapted clinical guideline on overactive bladder (OAB) for the Iranian context, which was produced by my evidence-based medicine (EBM) research centre in Tabriz, Iran. Our adapted guideline includes recommendations about initial assessment and conservative, pharmacological and surgical management.
For a healthcare professional, these recommendations may be relatively easy to interpret, but for a patient, they can be quite difficult to understand. Taking the OAB guideline as an example, a patient may ask, What does ‘urodynamic testing is not routinely recommended for patients with uncomplicated overactive bladder, when prescribing first-line treatment’ mean? They may want to know, What does ‘strong evidence’ actually mean? Or what exactly does it mean when a treatment ‘may be considered’?
For healthcare professionals, if a recommendation says that a treatment has potential benefits but also important risks, how can we explain this to the patient clearly to ensure understanding, while not making the decision for the patient, and making it clear that the decision rests with them?
I asked AI (ChatGPT 4.5) to help transform the clinical recommendations into a patient-friendly summary, and the result was surprisingly good in some ways. The information was organised into questions that a patient might actually ask:
- Do I need special tests?
- What can I do myself?
- What medicines are available in Iran?
- What happens if medicines do not work?
- What should I know before Botox treatment?
- What other treatment options are available?
The language was much easier to read than the original guideline, and this is one of the things I find exciting about AI. AI can help us move from ‘evidence that exists’ to ‘evidence that people can actually understand’. But then I started looking more carefully at the output, and this is when I realised that maybe the result wasn’t so good after all.
Making evidence simpler can also change it
The guideline says that there is weak evidence supporting the use of nutritional products, vitamins, supplements or herbal medicines for controlling OAB symptoms. At first glance, it seems simple enough. When AI changes this into ‘Supplements are not recommended for overactive bladder’, it sounds clear, and a patient can easily understand it. But, of course, it is not the same message. ‘Weak evidence’ does not mean ‘the treatment does not work’. It means that the available evidence is not strong enough to give us confidence about its effects. This may look like a small difference in wording, but for evidence-based healthcare, it is not a small difference. The level of certainty matters.
Another example is the recommendation about anticholinergic medicines. The guideline asks clinicians to inform patients about the possible risk of dementia and cognitive impairment. When AI simplifies this to ‘Anticholinergic medicines cause dementia’, the sentence is much stronger than the original recommendation. AI may produce a sentence that sounds more direct and more confident, while the evidence itself may be uncertain or conditional. It made me realise that communicating trustworthy evidence is not simply about making scientific language easier. It is about making it easier to understand while communicating the meaning accurately.
What should AI preserve?
When we communicate a clinical guideline to patients, I think there are several things that should not be lost during simplification. The first is certainty. Words such as ‘strong evidence’, ‘weak evidence’, ‘may’, ‘should’, ‘should not’, and ‘may be considered’ are not just writing choices. They communicate our level of confidence and the strength of a recommendation.
The second element we should not lose is conditions and exceptions. For example, the guideline does not simply say that patients with OAB do not need urodynamic testing. It says that urodynamic testing is not routinely recommended when uncomplicated OAB is being treated with the first-line treatment. Additional testing may be considered when the diagnosis is uncertain. Removing the second part can make the first recommendation sound much more absolute than it really is.
The third element we have to retain is benefits and harms. This is particularly important for treatments such as Botox injections. A patient needs to know not only that Botox may improve their symptoms but also about the possibility of urinary tract infection, problems with bladder emptying, the possible need for intermittent catheterisation and uncertainty about long-term effects. A shorter summary is not necessarily a better summary if important risks disappear.
The fourth factor that must stay is patient choice. Some clinical decisions are not simply about whether a treatment works. They are about whether the expected benefits, risks, burden and uncertainty are acceptable to that particular patient. AI should help patients understand the choice that they can make. It should not quietly make the choice for them.
So, can we trust AI to communicate evidence?
My answer is not simply yes or no. I think AI can be a very useful tool for communicating evidence. It can help researchers and healthcare professionals create plain-language summaries, translate information into different languages, organise complex recommendations and adapt communication to different audiences. But I do not think we should treat an AI-generated summary as automatically trustworthy just because it sounds clear, professional and convincing.
Clear language is not the same as trustworthy information
For me, the more important question than ‘Did AI produce a good summary?’ is ‘Does the summary still faithfully represent the evidence?’ This requires human review. When I checked the AI-generated patient summary against the original adapted guideline, I found that the most important issues were not necessarily clear factual errors. They were subtle changes in meaning; for example, a recommendation became stronger, a condition was removed or an additional piece of advice appeared even though it was not part of the original adapted guideline. These changes can easily be missed because the final text may sound completely reasonable. This is why I believe that human oversight remains essential in AI-assisted evidence communication.
What does this mean for patients?
There is another side of this discussion that I think deserves more attention, because patients are already using AI to understand health information. They may ask AI about symptoms, laboratory results, medicines, treatment options or information they have received from their doctors. For many people, AI may become one of the first places they go when they have a health question.
This creates a new opportunity for health literacy. Imagine a patient asking, ‘What are my treatment options for overactive bladder?’ A good AI system could potentially provide understandable information based on trustworthy guidelines and explain the options in plain language. But patients also need to know how to question the answer. Where did this information come from? Is it based on a guideline, a systematic review or something else? How certain is the evidence? Are there risks or alternatives that have not been mentioned? Does this information apply to my situation?
In other words, in an AI-mediated world, health literacy increasingly needs to include the ability to critically engage with AI-generated health information. Researchers, healthcare professionals, guideline developers, policymakers, technology developers and patient organisations all have a role to play.
From producing evidence to communicating evidence
Working with evidence has taught me that producing good evidence is only the beginning. If evidence remains inside a 200-page guideline, it may not reach the people who need it. We need to communicate it, translate it and make it accessible. And now, we also need to think about how AI can help us do this. But accessibility should not come at the cost of trustworthiness. For me, this is where the future of evidence communication lies.
I do not see AI as a replacement for researchers, clinicians or patients’ judgement. I see it as a tool that can help us communicate evidence more effectively, especially when the amount and complexity of information continue to grow.
But the final responsibility for trustworthy communication cannot simply be handed over to an algorithm. Perhaps the goal should not be to ask AI to make evidence shorter. Perhaps we should ask AI to make evidence more understandable while keeping its uncertainty, context, limitations, benefits and harms visible. That is a much harder task, but is also a much more important one.
Key messages
- AI can make recommendations from a guideline easier to understand, but simplification should not change their meaning. Uncertainty, conditions, exceptions, benefits and harms need to remain visible
- AI-generated health information needs a human review. A response that sounds clear and convincing is not necessarily faithful to the original evidence
- We need to build AI and evidence literacy. Patients and the public should be supported to ask where health information comes from, how certain it is and whether it applies to their situation
To link to this article - DOI: https://doi.org/10.70253/BQYI5116
Disclaimer
The views expressed in this World EBHC Day Blog, as well as any errors or omissions, are the sole responsibility of the author and do not represent the views of the World EBHC Day Steering Committee, Official Partners or Sponsors; nor does it imply endorsement by the aforementioned parties.
