The REPRISE project: protocol for an evaluation of REProducibility and Replicability In Syntheses of Evidence

Systematic Reviews

David Nunan¹  David Moher²  Sathya Karunananthan³  Shinichi Nakagawa⁴  Lara J. Maxwell⁵  Neal R. Haddaway⁶  Julian P. T. Higgins⁷  Daniel G. Hamilton⁸  Fiona M. Fidler⁸  Peter Tugwell⁹  Vivian A. Welch⁹  Matthew J. Page^1,10  Raju Kanukula^1,10  Joanne E. McKenzie^1,10  Steve McDonald^1,10  Sue E. Brennan^1,10 

[1] Centre for Evidence-Based Medicine, Oxford University;Centre for Journalology, Clinical Epidemiology Program, Ottawa Hospital Research Institute;Clinical Epidemiology Program, Ottawa Hospital Research Institute;Evolution & Ecology Research Centre and School of Biological, Earth and Environmental Sciences, University of New South Wales;Faculty of Medicine, University of Ottawa;Mercator Research Institute on Global Commons and Climate Change;Population Health Sciences, Bristol Medical School, University of Bristol;School of BioSciences, University of Melbourne;School of Epidemiology and Public Health, Faculty of Medicine, University of Ottawa;School of Public Health and Preventive Medicine, Monash University;

关键词: Reproducibility of Results;    Replication;    Transparency;    Systematic reviews;    Meta-analysis;    Methodology;   

DOI  :  10.1186/s13643-021-01670-0

来源: DOAJ

【 摘 要 】

Abstract Background Investigations of transparency, reproducibility and replicability in science have been directed largely at individual studies. It is just as critical to explore these issues in syntheses of studies, such as systematic reviews, given their influence on decision-making and future research. We aim to explore various aspects relating to the transparency, reproducibility and replicability of several components of systematic reviews with meta-analysis of the effects of health, social, behavioural and educational interventions. Methods The REPRISE (REProducibility and Replicability In Syntheses of Evidence) project consists of four studies. We will evaluate the completeness of reporting and sharing of review data, analytic code and other materials in a random sample of 300 systematic reviews of interventions published in 2020 (Study 1). We will survey authors of systematic reviews to explore their views on sharing review data, analytic code and other materials and their understanding of and opinions about replication of systematic reviews (Study 2). We will then evaluate the extent of variation in results when we (a) independently reproduce meta-analyses using the same computational steps and analytic code (if available) as used in the original review (Study 3), and (b) crowdsource teams of systematic reviewers to independently replicate a subset of methods (searches for studies, selection of studies for inclusion, collection of outcome data, and synthesis of results) in a sample of the original reviews; 30 reviews will be replicated by 1 team each and 2 reviews will be replicated by 15 teams (Study 4). Discussion The REPRISE project takes a systematic approach to determine how reliable systematic reviews of interventions are. We anticipate that results of the REPRISE project will inform strategies to improve the conduct and reporting of future systematic reviews.

【 授权许可】

Unknown