AI Sorted, Humans Decided
Author: Bilal Ahmad Khan1
1. Pakistan Institute of Living and Learning
First Attempt In Learning: AI Literacy, Not AI Dependency
Every research journey begins long before the first publication. Mine began with a question mentors, not machines, helped me answer: Where do my strengths truly lie? This year’s World EBHC Day theme, Evidence and AI: People at the Centre, reminds us that the tools are powerful, but the choices are ours. Technology accelerated my work; people gave it direction.
The chain continues: from communicating evidence to creating it
Last year, in Breaking Barriers, Training Champions: My Journey in Collaborative Knowledge Communication, I wrote about how evidence is communicated, and committed to training 100 more researchers in Pakistan. Herein, I will consider how evidence is produced in the age of artificial intelligence, and why the people in that process matter more than ever.
The rejection that redirected me: where it all began
My route into evidence synthesis was built by leaders, not platforms. At the International Islamic University, Islamabad (IIUI), my supervisors, Dr Irshad and Dr Mujeeb, saw it before I did: ‘Your strength is research.’ That encouragement shaped everything after. During my MS, I interviewed stakeholders for the HEC-funded project, Challenges in Mental Health, Law and Human Rights in Pakistan (HEC (FD)/2017/13642), and co-authored the article Stakeholders’ Perspective on Mental Health Laws in Pakistan: A Mixed Method Study. These activities taught me my first lesson: evidence is not an abstraction; it begins and ends with people’s lived experiences. The NIHR IMPACT trial (17/63/130) at the Institute of Psychiatry, DiaDeM trial (NIHR200806) and an MS in Epidemiology and Biostatistics at Khyber Medical University (KMU) then taught me the craft: Covidence, Rayyan, meta-analysis in R, Stata, RevMan and reviews of underweight, overweight and obesity in people with severe mental illness.
In January 2023, as a biostatistician at the KMU Clinical Trials Unit, I applied for a NIHR Short Placement Award for Research Collaboration (SPARC), to master evidence synthesis and bring those skills home to Pakistan and Afghanistan. I was not selected.
I was disheartened until my father, Mr Sohail Ahmad, shared Dr A.P.J. Abdul Kalam’s words: F.A.I.L. means First Attempt In Learning; E.N.D. means Effort Never Dies; and N.O. means Next Opportunity. At the heart of SPARC is a beautiful obligation: whatever an awardee learns must not stay with them; it must be transferred back as training in their home country. I held myself to that standard. If I could not travel to an opportunity, I would build one in Peshawar, where talent is abundant but direction is scarce. This goal did not require a plane ticket. It required intention.
Building the capacity I once hoped to receive
I first heard of the Covidence Student Ambassador Program from Dr Aatik Arsh, a 2022 Covidence Student Ambassador from KMU. In 2023, I applied and was honoured to be one of 22 ambassadors selected worldwide, representing Pakistan. Instead of the single session required of me, I delivered workshops for early career researchers at the “International Islamic University Islamabad”, “Baqai Institute of Diabetology and Endocrinology, Karachi”, “Institute of Psychiatry, Benazir Bhutto Hospital, Rawalpindi”, “Khyber Medical University, Peshawar” and “Riphah International University, Islamabad”. I have since mentored doctoral researchers at KMU through reviews on childhood vaccine uptake in LMICs, tobacco cessation in dental settings in South Asia and integrated tuberculosis–diabetes care in South Asian Association for Regional Cooperation countries, and contributed to reviews of recruitment and retention of hard-to-reach, marginalised communities, cognitive behavioural therapy for harmful cannabis use and childhood obesity in LMICs. Every skill we acquire carries a duty of transfer: learn, then teach; receive, then give.
Evidence and AI but people at the centre
The review closest to this year’s theme is one I led, Coping Strategies and Resilient Behavior among Frontline Healthcare Workers: A Scoping Review, which was published in Dialogues in Health. Following JBI methodology and PRISMA-ScR guidelines, we searched five major databases from inception to December 2024 to map how frontline healthcare workers cope with stress and build resilience. The foundations for this review were laid by the human review team. Of 2,407 records identified, 512 duplicates were removed in Rayyan.ai, and then two teams of human reviewers independently screened all titles, abstracts and full texts, resolving disagreements by consensus, which left 75 included articles.
We integrated artificial intelligence only in our analysis phase. Claude 3.5 Sonnet, a large language model developed by Anthropic, helped us manage and analyse the extracted data: organising studies by World Bank income classifications, identifying intervention types, outcome measures and recurring coping patterns and keeping terminology consistent. This sharply reduced the time to initial pattern recognition compared with traditional manual coding. For a small team in a low- and middle-income country, where reviewer time is the scarcest resource of all, this efficiency was not a luxury but an equaliser. At every decision point, though, humans stayed at the centre. Two senior researchers independently reviewed every theme generated with AI’s assistance, discussing and resolving discrepancies. That oversight proved essential. Across our 75 studies, AI placed 27 in the lower-middle-income group, where only 20 belonged, conflating interventions that looked alike but sat in different economies. Furthermore, it recorded no mobile-based interventions in LMIC settings, seeing only sophisticated web platforms, until we identified the basic digital tools and smartphone apps it had passed over. And it could not judge clinical significance, cultural nuance or implementation feasibility. Rather than hide this, we published an AI–human comparison table in the paper itself, showing exactly where the AI erred and how human expertise corrected it. AI sorted; humans decided.
AI-literate people, not AI-dependent people
That is the message I now carry into every training session. I remember when screening in our setting meant Excel spreadsheets and duplicates spotted line by line, with weeks gone before the real intellectual work began. Today, that is a single click, screening and extraction sit in one platform and large language models open up possibilities we did not have before. Knowing AI and using it well are different things. AI literacy means understanding what a tool is doing, verifying its output, documenting its errors honestly and directing it with human judgement. AI dependency means surrendering that judgement. The first lets researchers in LMICs compete with the best in the world; the second quietly erodes the rigour on which evidence-based healthcare was built.
Our Global Evidence Summit 2024 poster in Prague counted the six million hours spent on title and abstract screening every year: time we could return to patients, communities and capacity building if we let AI do what it does best.
What this means in practice
Four commitments have emerged from this work, for anyone using AI in evidence synthesis:
- Put AI after the human foundations, not before them: screening and eligibility stayed with two independent human reviewers; AI entered at the synthesis stage, where volume, not judgement, was the bottleneck.
- Report AI use the way you report your methods: name the model and version, state precisely what it did and publish an AI–human comparison, as we did, so readers can see where it failed.
- Read the AI’s errors as evidence: in our published paper, the model placed 27 studies in the LMIC group, where only 20 belonged, which illustrates that every AI output needs a human check.
- Spend the time AI gives back on the work only people can do: patient and community involvement, contextual interpretation and training for the next reviewer.
Lessons learned from my journey
From a rejected application, to Covidence Ambassador, and on to leading a department of evidence synthesis at PILL, I have learned one thing: if there’s no opportunity, create it, then give it away, because knowledge that is not transferred is knowledge half-wasted. A closed door is not the end of the road; it is an invitation to construct a new one.
Gratitude: every leader is built by leaders
No one becomes a leader alone. I am grateful to Prof Kamran Siddiqi, Prof Najma Siddiqi and Dr Saima Afaq, who guided me towards the opportunity of the MS in Epidemiology and Biostatistics, Dr Gerardo Zavala, who trained me in systematic review and meta-analysis, and Prof Saeed Farooq, Dr Zohaib Khan, Dr. Naheed Mahsood, Dr Faiza Aslam, Dr Saima Aleem, Prof Muhammad Tahir Khalily, Dr Nauman Arif and Razia Aliani, who opened the doors of AI literacy training to me. Their guidance in the world of research, evidence synthesis and artificial intelligence has been invaluable. Moreover, my special and heartfelt gratitude goes to Prof Rod Taylor and Prof Nusrat Husain, my mentors, who gave me the right direction and supported my path towards global health.
And to my family: my mother’s prayers, my father’s guidance and my siblings’ encouragement carried me through the hardest chapters and kept me disciplined and motivated.
Faith in our purpose, unity with our people and discipline in our methods: Pakistan’s national motto, ‘Unity, Faith, Discipline’, is as good a guide to AI in evidence synthesis as any. If we hold to it, AI will remain what it should always be: a powerful tool in human hands, with people, always, at the centre.
References
Afzal, M., Siddiqi, N., Ahmad, B., Afsheen, N., Aslam, F., Ali, A., Ayesha, R., Bryant, M., Holt, R., Khalid, H., Ishaq, K., Koly, K. N., Rajan, S., Saba, J., Tirbhowan, N., & Zavala, G. A. (2021). Prevalence of overweight and obesity in people with severe mental illness: Systematic review and meta-analysis. Frontiers in Endocrinology, 12, 769309. https://doi.org/10.3389/fendo.2021.769309
Ahmad, B., Rehman, K., Bangash, S., Zarkaish, R., Babak, M., Rehman, S., Ihsan, S., Khattak, F. A., Irfan, M., Rehman, A., Khan, A. J., & Khan, Z. (2025). Coping strategies and resilient behavior among frontline healthcare workers: A scoping review. Dialogues Health, 7, 100252. https://doi.org/10.1016/j.dialog.2025.100252
Aleem, S., Afaq, S., Gill, S., Ahmad, B., Michael, Z., Zarkaish, R., & Khan, Z. (2026). Implementation determinants of integrated tuberculosis and diabetes care in South Asian Association for Regional Cooperation (SAARC) countries: A systematic review. International Journal of Integrated Care, 26, 21. https://doi.org/10.5334/ijic.9882
Aleem, S., Ahmad, B., Zala, Khalid Qazi, F., Ullah, S., Ullah, A., Ul Haq, H., Sultan, S., Rahim, M., Zarkaish, R., Abdul Jalil, S., Ahmad, F., Kibria, Z., & Khan, Z. (2025). Recruitment methods and retention in research studies of hard-to-reach communities marginalised due to discrimination, and social stigmatization: A systematic review. Journal of Gandhara Medical and Dental Sciences, 13, 117–129. https://doi.org/10.37762/jgmds.13-1.837
Brown, J., Khan, B., Ashkar, I., Basheer, S., Munir, M., & Elliott, J. (2024). Untapped potential for systematic review software and automation to reduce the 6 million hours spent on title/abstract screening annually. Cochrane Colloquium Abstracts. https://www.covidence.org/resource/untapped-potential-for-systematic-review-software-automation/
Hakim, M., Afaq, S., Imtiaz, S., Ali, F., Khan, B. A., & Haq, Z. U. (2025). Strategies for enhancing vaccine uptake among children under two years age in low- and middle-income countries (LMICs): A scoping review protocol. OSF. https://doi.org/10.17605/OSF.IO/M3WUK
Kabir, S., Ahmad, B., Zahid, M., Aleem, S., Nooreen, S., Waheed, Z., Rasool, S., Irfan, M., & Khan, Z. (2026). Non-pharmacological interventions for tobacco cessation delivered in dental settings in South Asia: A scoping review. Discover Public Health, 23, 872. https://doi.org/10.1186/s12982-026-02281-7
Khalily, M. T., Rehman, A. U., Bhatti, M. M., Hallahan, B., Ahmad, I., Mehmood, M. I., Khan, S. H., & Khan, B. A. (2021). Stakeholders’ perspective on mental health laws in Pakistan: A mixed method study. International Journal of Law and Psychiatry, 74, 101647. https://doi.org/ 10.1016/j.ijlp.2020.101647
Khan, B. A. (2023a). Leading with passion: My journey with Covidence as a student ambassador. Covidence Blog. https://www.covidence.org/blog/leading-with-passion-my-journey-with-covidence-as-a-student-ambassador/
Khan, B. A. (2023b). Streamlining research: Unveiling the power of evidence synthesis with Covidence. Covidence Blog. https://www.covidence.org/blog/streamlining-research-unveiling-the-power-of-evidence-synthesis-with-covidence/
Khan, B. A. (2025). Breaking barriers, training champions: my journey in collaborative knowledge communication. World EBHC Day. https://doi.org/10.70253/OBWJ6595
Khan, B. A., Khalid, H., Siddiqi, N., Aslam, F., Ayesha, R., Afzal, M., Rajan, S., Appuhamy, K., Koly, K. N., Bryant, M., Holt, R. I. G., & Zavala, G. A. (2023). Prevalence of underweight in people with severe mental illness: Systematic review and meta‐analysis. Mental Health Science, 1, 10–22. https://doi.org/10.1002/mhs2.7
Tricco, A. C., Lillie, E., Zarin, W., O'Brien, K. K., Colquhoun, H., Levac, D., Moher, D., Peters, M. D. J., Horsley, T., Weeks, L., Hempel, S., Akl, E. A., Chang, C., McGowan, J., Stewart, L., Hartling, L., Aldcroft, A., Wilson, M. G., Garritty, C., Lewin, S., … Straus, S. E. (2018). PRISMA Extension for Scoping Reviews (PRISMA-ScR): Checklist and explanation. Annals of Internal Medicine, 169, 467–473. https://doi.org/10.7326/M18-0850
Ullah, S., Ullah, A., Ahmad, F., Latif, A., Sohaib, M., & Khan, M. F. (2026). Effectiveness of cognitive behavioral therapy for harmful cannabis use: A systematic review and meta-analysis. Cognitive Behaviour Therapy, 1–21. https://doi.org/10.1080/16506073.2026.2613114
Zavala, G. A., Asim, M., Ayub, A., Ahmad, A. M. R., Pillai, A. P., Pavlou, E., Sharma, A., Khan, B. A., Shakoor, H., Mansukoski, L., Obregon, O. P. G., Khan, U., Khan, Z., Rehman, A., Nisa, B. U., Bibi, H., Iqbal, R., Gill, S., Ashfaq, U., khan, F. U., ... Afaq, S. (2025). PROTOCOL: Effectiveness of interventions for the prevention and treatment of obesity in children and adolescents from low- and middle-income countries: A systematic review and meta-analysis. Campbell Systematic Reviews, 21, e70081. https://doi.org/10.1002/cl2.70081
To link to this article - DOI: https://doi.org/10.70253/KQEM9198
Disclaimer
The views expressed in this World EBHC Day Blog, as well as any errors or omissions, are the sole responsibility of the author and do not represent the views of the World EBHC Day Steering Committee, Official Partners or Sponsors; nor does it imply endorsement by the aforementioned parties.
