Collating the knowledge base for core outcome set development: developing and appraising the search strategy for a systematic review

BMC Medical Research Methodology

Mike Clarke²  Paula R Williamson¹  Elizabeth Gargon¹ 

[1] Department of Biostatistics, University of Liverpool, Liverpool, UK;All Ireland Hub for Trials Methodology Research, Queen’s University Belfast, Belfast, UK

关键词: Medical research;    Systematic review;    Resources;    Database;    Core outcome set;   

Others  :  1177565
DOI  :  10.1186/s12874-015-0019-9

 received in 2014-07-22, accepted in 2015-03-17,  发布年份 2015

【 摘 要 】

Background

The COMET (Core Outcome Measures in Effectiveness Trials) Initiative is developing a publicly accessible online resource to collate the knowledge base for core outcome set development (COS) and the applied work from different health conditions. Ensuring that the database is as comprehensive as possible and keeping it up to date are key to its value for users. This requires the development and application of an optimal, multi-faceted search strategy to identify relevant material. This paper describes the challenges of designing and implementing such a search, outlining the development of the search strategy for studies of COS development, and, in turn, the process for establishing a database of COS.

Methods

We investigated the performance characteristics of this strategy including sensitivity, precision and numbers needed to read. We compared the contribution of databases towards identifying included studies to identify the best combination of methods to retrieve all included studies.

Results

Recall of the search strategies ranged from 4% to 87%, and precision from 0.77% to 1.13%. MEDLINE performed best in terms of recall, retrieving 216 (87%) of the 250 included records, followed by Scopus (44%). The Cochrane Methodology Register found just 4% of the included records. MEDLINE was also the database with the highest precision. The number needed to read varied between 89 (MEDLINE) and 130 (SCOPUS).

Conclusions

We found that two databases and hand searching were required to locate all of the studies in this review. MEDLINE alone retrieved 87% of the included studies, but actually 97% of the included studies were indexed on MEDLINE. The Cochrane Methodology Register did not contribute any records that were not found in the other databases, and will not be included in our future searches to identify studies developing COS. SCOPUS had the lowest precision rate (0.77) and highest number needed to read (130). In future COMET searches for COS a balance needs to be struck between the work involved in screening large numbers of records, the frequency of the searching and the likelihood that eligible studies will be identified by means other than the database searches.

【 授权许可】

   
2015 Gargon et al.; licensee BioMed Central.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Tovey D: Impact of Cochrane Reviews. Cochrane Database Syst Rev 2010., 2011Article ID ED0000007
• [2]Hutton JL, Williamson PR: Bias in meta-analysis due to outcome variable selection within studies. Appl Stat 2000, 49:359-70.
• [3]Dwan K, Altman DG, Arnaiz JA, Bloom J, Chan A-W, Cronin E, et al.: Systematic review of the empirical evidence of study publication bias and outcome reporting bias. PLoS One 2008, 3:e3081.
• [4]Kirkham JJ, Dwan KM, Altman DG, Gamble C, Dodd S, Smyth R, et al.: The impact of outcome reporting bias in randomised controlled trials on a cohort of systematic reviews. BMJ 2010, 340:c365.
• [5]Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al.: Developing core outcome sets for clinical trials: issues to consider. Trials 2012, 13:132. BioMed Central Full Text
• [6]Gargon E, Gurung B, Medley N, Altman DG, Blazeby JM, Clarke M, et al.: Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One 2014, 9:e99111.
• [7]Sinha I, Jones L, Smyth RL, Williamson PR: A systematic review of studies that aim to determine which outcomes to measure in clinical trials in children. PLoS Med 2008, 5:e96.
• [8]Sinha IP, Smyth RL, Williamson PR: Using the delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med 2011, 8:e1000393.
• [9]Oliver S, Gray J: A bibliography of research reports about patients’, clinicians’ and researchers’ priorities for new research. James Lind Alliance, London; 2006.
• [10]Hill B: Comparison of journal title coverage between CINAHL and Scopus. J Med Libr Assoc 2009, 97:313-4.
• [11]Watson RJ, Richardson PH: Identifying randomized controlled trials of cognitive therapy for depression: comparing the efficiency of Embase, Medline and PsycINFO bibliographic databases. Br J Med Psychol 1999, 72(Pt 4):535-42.
• [12]Wong SS, Wilczynski NL, Haynes RB: Comparison of top-performing search strategies for detecting clinically sound treatment studies and systematic reviews in MEDLINE and EMBASE. J Med Libr Assoc 2006, 94:451-5.
• [13]Hopewell S, Clarke Mike J, Lefebvre C, Scherer Roberta W: Handsearching versus electronic searching to identify reports of randomized trials. In Cochrane Database of Systematic Reviews. John Wiley & Sons, Ltd, UK; 2007.
• [14]Kirkham JJ, Gargon E, Clarke M, Williamson PR: Can a core outcome set improve the quality of systematic reviews?–a survey of the Co-ordinating Editors of Cochrane Review Groups. Trials 2013, 14:21. BioMed Central Full Text
• [15]Higgins JPT, Green S. Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2 [updated September 2009]. The Cochrane Collaboration 2009. Available from www.cochrane-handbook.org.
• [16]Clarke M: Searching MEDLINE for randomised trials. BMJ 1993, 307:565.
• [17]Dickersin K, Manheimer E, Wieland S, Robinson KA, Lefebvre C, McDonald S: Development of the Cochrane Collaboration’s CENTRAL Register of controlled clinical trials. Eval Health Prof 2002, 25:38-64.
• [18]Lefebvre C, Eisinga A, McDonald S, Paul N: Enhancing access to reports of randomized trials published world-wide–the contribution of EMBASE records to the Cochrane Central Register of Controlled Trials (CENTRAL) in The Cochrane Library. Emerg Themes Epidemiol 2008, 5:13. BioMed Central Full Text
• [19]Miller AB, Hoogstraten B, Staquet M, Winkler A: Reporting results of cancer treatment. Cancer 1981, 47:207-14.
• [20]SRNT Subcommittee on Biochemical Verification: Biochemical verification of tobacco use and cessation Nicotine Tob Res 2002, 4:149-59.
• [21]Schlaepfer TE, Fins JJ: Deep brain stimulation and the neuroethics of responsible publishing: when one is not enough. JAMA 2010, 303:775-6.
• [22]McAllister M, Davies L, Payne K, Nicholls S, Donnai D, MacLeod R: The emotional effects of genetic diseases: implications for clinical genetics. Am J Med Genet A 2007, 143A:2651-61.
• [23]McAllister M, Dunn G, Todd C: Empowerment: qualitative underpinning of a new clinical genetics-specific patient-reported outcome. Eur J Hum Genet 2011, 19:125-30.
• [24]McAllister M, Payne K, Macleod R, Nicholls S, Dian D, Davies L: Patient empowerment in clinical genetics services. J Health Psychol 2008, 13:895-905.
• [25]McAllister M, Payne K, Nicholls S, MacLeod R, Donnai D, Davies LM: Improving service evaluation in clinical genetics: identifying effects of genetic diseases on individuals and families. J Genet Couns 2007, 16:71-83.
• [26]McAllister M, Wood AM, Dunn G, Shiloh S, Todd C: The Genetic Counseling Outcome Scale: a new patient-reported outcome measure for clinical genetics services. Clin Genet 2011, 79:413-24.
• [27]Payne K, Nicholls S, McAllister M, Macleod R, Donnai D, Davies LM: Outcome measurement in clinical genetics services: a systematic review of validated measures. Value Health 2008, 11:497-508.