Can AI Bridge the Evidence–Practice Gap? Lessons from EviNurse
Author: Weijie Xing1
1. JBI Fudan University Centre for Evidence-Based Nursing
The problem is not a lack of evidence
“Is there any new evidence on managing chemotherapy-induced nausea?”
This simple question came from a nurse during a discussion with frontline clinicians.
The question itself was not difficult to answer, but the nurse made the point that it was challenging for them to find a trustworthy answer during a busy shift.
They added,
“I wish all the evidence could be in one place, so I could simply ask a question and get a trustworthy answer.”
The challenge that the nurse touched on that day was not a lack of evidence, but making evidence accessible, usable, and relevant at the point of care.
That conversation became one of the starting points for developing EviNurse.
Can AI help bridge the evidence–practice gap?
Artificial intelligence offers an exciting possibility. Large language models such as ChatGPT and Gemini can understand natural language, search and summarise large amounts of information, and respond to specific questions in seconds.
But in healthcare, fluent answers are not enough. Answers also need to be accurate, transparent, evidence-based, and supported by valid references. Trustworthy evidence matters more than persuasive language.
What is EviNurse, and how does it bridge the evidence–practice gap?
Over the past two years, our team has explored the issue that clinicians face in accessing trustworthy evidence through the development of EviNurse, a large language model designed specifically to support evidence-informed nursing practice.
We first constructed NursData, a curated nursing evidence database. It contains evidence summaries, clinical guidelines, systematic reviews, original studies, textbooks, and policy documents. Importantly, these resources are organised according to the 5S evidence pyramid, ensuring that higher-level evidence is prioritised whenever available.
We then combined this evidence database with a large language model using supervised fine-tuning and retrieval-augmented generation.
In practice, this means that when a nurse asks a question, EviNurse does not rely only on what the model has memorised during training. Instead, it first retrieves the most relevant evidence from NursData, prioritises higher-level evidence, and then generates a response grounded in those sources. Whenever possible, it also provides the original references, allowing users to verify the origins of its recommendations.
This design changes the way nurses interact with evidence. Instead of expecting users to search for evidence themselves, EviNurse helps bring relevant evidence into the clinical conversation and shows the sources behind each answer.
How did EviNurse perform?
Developing EviNurse was only the first step. We also wanted to know whether it performed better than general-purpose AI models and, more importantly, whether nurses would actually find it useful.
Rather than relying on a single benchmark, we evaluated EviNurse in three stages.
First, we tested its fundamental nursing knowledge. We asked EviNurse and six leading general-purpose AI models to answer more than 3,400 multiple-choice nursing questions, allowing us to compare their performance on standard nursing knowledge.
Second, we evaluated how well it answered realistic nursing scenario questions. We developed 25 short-answer questions covering five nursing specialties and asked exactly the same questions, using the same prompts, to EviNurse and the six general-purpose models. All responses were anonymised before being independently reviewed by 15 experienced nursing experts, who rated each response for accuracy, completeness, safety, reliability, and overall satisfaction.
Finally, we tested its usability in practice. We invited 316 nurses from all provinces of mainland China to use EviNurse for one week. Each nurse interacted with the system at least 10 times before completing a usability questionnaire covering model quality, information quality, interface quality, user satisfaction, intention to use, and perceived benefits.
The findings were encouraging.
In the automated evaluation, EviNurse achieved 91.45% accuracy across more than 3,400 nursing multiple-choice questions. It ranked second among the seven models, behind Gemini 3 Flash Preview (94.33%).
In the expert evaluation, EviNurse outperformed all six general-purpose models. Nursing experts rated its responses highest for reliability and overall satisfaction, suggesting that it was particularly effective at providing trustworthy, evidence-informed answers to realistic nursing questions.
After one week of use, 316 nurses evaluated EviNurse positively, particularly for the quality of the information it provided, their willingness to continue using the tool, and its perceived benefits for clinical practice.
Together, these findings suggest that grounding AI in high-quality evidence can improve its trustworthiness beyond that of general-purpose models. They also demonstrate that such systems can help nurses access, interpret, and apply the best available evidence in everyday practice.
What we learned from building EviNurse
Lesson 1: Evidence must come before answers
Many AI systems are trained on large amounts of internet content. While this approach can produce broad knowledge, it does not guarantee that responses are based on the best available evidence.
One of the most important lessons we learned from developing EviNurse was that evidence quality matters as much as model capability.
By organising knowledge according to the 5S evidence pyramid, EviNurse was designed to prioritise higher-level evidence whenever available. This helps ensure that recommendations are based on stronger and more synthesised evidence rather than isolated studies or unverified information.
Lesson 2: Transparency builds trust
Healthcare professionals are trained to ask an important question: “What is the evidence behind this recommendation?” AI systems should be able to answer that question.
One of our goals was, therefore, to provide references alongside responses whenever possible. During evaluation, we found that EviNurse consistently provided authentic references, allowing users to see the origins of recommendations.
For evidence-based healthcare, helping users examine the evidence is often more valuable than simply asking them to trust the answer.
Lesson 3: End-users must be involved
Perhaps the most important lesson was that healthcare AI cannot be developed in isolation. Nurses played a central role throughout the project:
- Identifying real-world information needs
- Curating evidence sources
- Designing evaluation questions
- Assessing model performance
- Testing usability in clinical settings
In other words, nurses were not passive recipients of technology. Instead, they were active partners in its development. This human-centred approach proved essential.
Lesson 4: Validation matters more than innovation
Healthcare is full of exciting AI announcements, while far fewer tools undergo rigorous evaluation. One of the biggest lessons from this project was the importance of testing AI systems in multiple ways.
We evaluated the model using:
- Automated testing with nursing examination questions
- Expert assessment of clinical scenarios
- Real-world usability testing among nurses nationwide
Interestingly, EviNurse did not achieve the highest score in the examination-style evaluation.
However, it performed best in expert assessment, particularly for reliability and user satisfaction.
This produced an important insight: Excelling in benchmark tests does not necessarily mean excelling in real clinical practice.
The true value of healthcare AI lies not in answering questions correctly, but in helping healthcare professionals apply evidence in complex, real-world situations.
Key take-home messages
1. The evidence–practice gap is often an access problem
Healthcare professionals need timely and practical access to trustworthy evidence, not simply more information.
2. Trustworthy AI requires trustworthy evidence
Transparency, evidence quality, and verifiable references are essential for responsible AI use in healthcare.
3. Human-centred design matters
Healthcare AI should be developed, evaluated, and implemented with active involvement from the people who will use these tools.
AI may help bridge the evidence–practice gap, but only if evidence is trustworthy, users are engaged, and people are at the centre of every stage of its development and implementation.
References
Alper, B. S., & Haynes, R. B. (2016). EBHC pyramid 5.0 for accessing preappraised evidence and guidance. BMJ Evidence-Based Medicine, 21(4), 123–125.
Bedi, S., Liu, Y., Orr-Ewing, L., Dash, D., Koyejo, S., Callahan, A., Fries, J. A., Wornow, M., Swaminathan, A., Lehmann, L. S., Hong, H. J., Kashyap, M., Chaurasia, A. R., Shah, N. R., Singh, K., Tazbaz, T., Milstein, A., Pfeffer, M. A., & Shah, N. H. (2025). Testing and evaluation of health care applications of large language models: A systematic review. JAMA, 333(4), 319–328.
Ruan, J., Dai, S., Bao, Z., Wei, Z., & Xing, W. (2026). Development and usability evaluation of a generative artificial intelligence-based nursing evidence question-answering model. Chinese Journal of Nursing, 61(13), 1733–1738.
Ruan, J., Tang, Y., Wei, Z., Xing, W., & Hu, Y. (2025). Nurses’ perspectives on evidence dissemination barriers and large language model–based support: Qualitative study using focus groups and nominal group technique. Journal of Medical Internet Research, 27, e80289.
To link to this article - DOI: https://doi.org/10.70253/TPYX6462
Disclaimer
The views expressed in this World EBHC Day Blog, as well as any errors or omissions, are the sole responsibility of the author and do not represent the views of the World EBHC Day Steering Committee, Official Partners or Sponsors; nor does it imply endorsement by the aforementioned parties.
