Teaching evidence-based medicine: finding reliable sources of information
‘Is this a good source?’ This is a common question from our medical students. Our students are in a new program that started in 2018, in which they do not study anatomy, physiology or embryology as some of us did in our training. Instead, they learn through problem-based learning (PBL) and early contact with clinical practice to develop entrustable professional activities (EPAs). In this model of teaching medicine, students are at the centre of retrieving information to meet their needs. As tutors, we guide them through this process in our Information Management Laboratory, which is embedded in the first three years of a six-year integrated curriculum. Here we describe some of the initial challenges that we found in our experience.
Background questions
Students in their first approach to a subject have many background questions, such as ‘How do contraceptives work?’ ‘What is heart failure?’ ‘How do vaccines work?’ Initially, we directed our students to the most obvious source: books. That is how many of us have studied medicine, seeing them as a reliable way to look for ‘filtered’ background information. However, students frequently use Google to search for explanations in sources most commonly used in the non-medical world, including Wikipedia, websites with information for patients and even news articles. In addition, they usually aim to ‘upgrade’ the trustability of the source by using questionable shortcuts, such as adding ‘PDF’ in their search boxes. As you can imagine, this usually leads to the use of many unreliable sources of information. For instance, when discussing birth control methods, a group of students found a PDF from ResearchGate claiming that contraceptives caused infertility (we are not linking the article to prevent the spread of misinformation). We tried to provide some preliminary guidance using the CRAAP test to identify reliable sources of information, but the students provided reasonable explanations as to how this source passed the test.
Foreground questions
As the students familiarise themselves with a topic, more specific questions emerge following a PICO format; for example, how does albuterol compare to placebo in children with bronchiolitis? What is the diagnostic accuracy of the MMSE test for dementia? In these instances, we use an evidence-based medicine (EBM) framework to develop skills for the core competencies in EBM. However, the students find many methods that we use to teach the searching and retrieval of studies to be outdated and ineffective for the timely response of their clinical questions for which they need quick answers. Teaching advanced search methods (MeSH terms, truncations, field tags, etc.) cannot match the friendliness of a quick Google search. Moreover, each platform is different and searching in Google will retrieve the results across platforms and databases, including preprints. As systematic reviewers, we feel more comfortable with a controlled search in PubMed, but this seems to be cumbersome from the students’ perspective.
Another focus of our training is to identify systematic reviews related to the PICO question. This used to be a sensible approach for busy students and clinicians who could not acquire, appraise and synthesise the finding of multiple studies addressing a single question among many. However, this has become a problematic approach as there seems to be an exponential growth in redundant and overlapping systematic reviews – usually of very low quality and many reaching different conclusions. One exception is perhaps those in The Cochrane Library, however, there are no Cochrane reviews for many foreground questions, especially non-therapeutic questions, including those answerable by qualitative evidence, which are becoming increasingly important to understand patients’ experiences, values and preferences.
Some ideas
We also need to acknowledge that the user experience with the sources of information is changing, and newer generations are used to different ways to find answers to their questions. An example of some activities that showed promising results for us included:
• Working on what works and doesn’t work when searching. For instance, promoting the identification of content from trusted databases using Google and avoiding shortcuts that are perceived to improve the identification of reliable data (such as adding ‘PDF’).
• Promoting the use of resources from our virtual library, as we see younger generations are less likely to use traditional textbooks in a physical format in a reading room at our library.
• Actively engaging in the use of Wikipedia (see the Cochrane-Wikipedia partnership), so students know how this resource is built, including the history of changes and the traceability of edits and sources used to support claims and understand its strengths and weaknesses.
• Focusing more on the critical appraisal of systematic reviews and how to interpret their findings, as they usually provide a good shortcut for EBM when they are of high quality.
Conclusion
As a group of healthcare professionals and educators, we face a rapidly changing scenario in which the newer generations use emerging formats to meet their information needs. The main challenges when teaching EBM are not related to how to use advanced search features in PubMed or how to apply a checklist for critical appraisal, but how to identify trustworthy sources of information in the context of low-quality research and fraudulent post-truth narratives, including anti-vaxxers and COVID-denialists. When teaching medical students at the start of their learning, we must bear in mind that they might behave more similarly to the general public than advanced students or health professionals with greater science literacy. So much of the day-to-day thinking on how we teach the sources of information may apply to earlier education and the general public. The earlier we combine the critical appraisal of the sources of information to the teaching of EBM, the greater chance we will have to fight misinformation.
Future steps
We need innovative approaches to teaching EBM in the context of the proliferation of accessible, yet sometimes unreliable, information. Furthermore, we need biomedical-information-literate students capable of evaluating information sources to resolve clinical questions with the diversity of available information. Finally, formal research is needed to improve how we teach EBM to the new generations, considering the changes in the world since this concept was first conceived.
Additional resources
Please see our initiatives at BMJ Evidence-Based Medicine aligned with the theme of evidence-based healthcare:
● Social media campaign during EBHC week, follow us on Twitter: @BMJ_EBM
● Call for papers: Addressing misinformation to promote evidence-informed healthcare
Juan Franco is the editor-in-chief of BMJ Evidence-Based Medicine, clinical editor at the BMJ, and full professor at Instituto Universitario Hospital Italiano de Buenos Aires (IUHIBA), Argentina. Luis Garegnani is associate editor of BMJ Evidence-Based Medicine, and Camila Escobar Liquitay is a Cochrane information specialist. We work as tutors at the Information Management Laboratory in the Medical School at IUHIBA.
Twitter handle @juan_francomd @micaelaescb
Competing interests: none declared (full statement of Juan Franco available here)
Disclaimer
The views expressed in this World EBHC Day Blog, as well as any errors or omissions, are the sole responsibility of the author and do not represent the views of the World EBHC Day Steering Committee, Official Partners or Sponsors; nor does it imply endorsement by the aforementioned parties.
