The automation of relevant trial registration screening for systematic review updates: an evaluation study on a large dataset of ClinicalTrials.gov registrations

BMC Medical Research Methodology

Didi Surian¹  Florence T. Bourgeois²  Adam G. Dunn³ 

[1] Centre for Health Informatics, Australian Institute of Health Innovation, Macquarie University, Sydney, NSW, Australia;Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA, USA;Department of Pediatrics, Harvard Medical School, Boston, MA, USA;Computational Health Informatics Program, Boston Children’s Hospital, Boston, MA, USA;The University of Sydney, Discipline of Biomedical Informatics and Digital Health, School of Medical Sciences, Faculty of Medicine and Health, 2006, Sydney, NSW, Australia;

关键词: Systematic reviews;    Trial registrations;    Document similarity;    Hierarchical clustering;   

DOI  :  10.1186/s12874-021-01485-6

来源: Springer

【 摘 要 】

BackgroundClinical trial registries can be used as sources of clinical evidence for systematic review synthesis and updating. Our aim was to evaluate methods for identifying clinical trial registrations that should be screened for inclusion in updates of published systematic reviews.MethodsA set of 4644 clinical trial registrations (ClinicalTrials.gov) included in 1089 systematic reviews (PubMed) were used to evaluate two methods (document similarity and hierarchical clustering) and representations (L2-normalised TF-IDF, Latent Dirichlet Allocation, and Doc2Vec) for ranking 163,501 completed clinical trials by relevance. Clinical trial registrations were ranked for each systematic review using seeding clinical trials, simulating how new relevant clinical trials could be automatically identified for an update. Performance was measured by the number of clinical trials that need to be screened to identify all relevant clinical trials.ResultsUsing the document similarity method with TF-IDF feature representation and Euclidean distance metric, all relevant clinical trials for half of the systematic reviews were identified after screening 99 trials (IQR 19 to 491). The best-performing hierarchical clustering was using Ward agglomerative clustering (with TF-IDF representation and Euclidean distance) and needed to screen 501 clinical trials (IQR 43 to 4363) to achieve the same result.ConclusionAn evaluation using a large set of mined links between published systematic reviews and clinical trial registrations showed that document similarity outperformed hierarchical clustering for identifying relevant clinical trials to include in systematic review updates.

【 授权许可】

CC BY