Guidance for using artificial intelligence for title and abstract screening while conducting knowledge syntheses

BMC Medical Research Methodology

Shannon E. Kelly¹  Candyce Hamel²  Mona Hersi²  George Wells³  Brian Hutton⁴  Ba’ Pham⁵  Sharon Straus⁶  Andrea C. Tricco⁷ 

[1] Cardiovascular Research Methods Centre, University of Ottawa Heart Institute, Ottawa, Ontario, Canada;School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada;Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada;Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada;Cardiovascular Research Methods Centre, University of Ottawa Heart Institute, Ottawa, Ontario, Canada;School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada;Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Ontario, Canada;School of Epidemiology and Public Health, University of Ottawa, Ottawa, Ontario, Canada;Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada;Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada;Department of Medicine, University of Toronto, Toronto, ON, Canada;Knowledge Translation Program, Li Ka Shing Knowledge Institute, St. Michael’s Hospital, Toronto, ON, Canada;Epidemiology Division and Institute for Health, Management, and Evaluation, Dalla Lana School of Public Health, University of Toronto, Toronto, Ontario, Canada;

关键词: Artificial intelligence;    Active machine-learning;    Best practice guidance;    Knowledge Synthesis;    Prioritization;    Title and abstract screening;   

DOI  :  10.1186/s12874-021-01451-2

来源: Springer

【 摘 要 】

BackgroundSystematic reviews are the cornerstone of evidence-based medicine. However, systematic reviews are time consuming and there is growing demand to produce evidence more quickly, while maintaining robust methods. In recent years, artificial intelligence and active-machine learning (AML) have been implemented into several SR software applications. As some of the barriers to adoption of new technologies are the challenges in set-up and how best to use these technologies, we have provided different situations and considerations for knowledge synthesis teams to consider when using artificial intelligence and AML for title and abstract screening.MethodsWe retrospectively evaluated the implementation and performance of AML across a set of ten historically completed systematic reviews. Based upon the findings from this work and in consideration of the barriers we have encountered and navigated during the past 24 months in using these tools prospectively in our research, we discussed and developed a series of practical recommendations for research teams to consider in seeking to implement AML tools for citation screening into their workflow.ResultsWe developed a seven-step framework and provide guidance for when and how to integrate artificial intelligence and AML into the title and abstract screening process. Steps include: (1) Consulting with Knowledge user/Expert Panel; (2) Developing the search strategy; (3) Preparing your review team; (4) Preparing your database; (5) Building the initial training set; (6) Ongoing screening; and (7) Truncating screening. During Step 6 and/or 7, you may also choose to optimize your team, by shifting some members to other review stages (e.g., full-text screening, data extraction).ConclusionArtificial intelligence and, more specifically, AML are well-developed tools for title and abstract screening and can be integrated into the screening process in several ways. Regardless of the method chosen, transparent reporting of these methods is critical for future studies evaluating artificial intelligence and AML.

【 授权许可】

CC BY