Impact of adding a limitations section to abstracts of systematic reviews on readers’ interpretation: a randomized controlled trial

BMC Medical Research Methodology

Isabelle Boutron³  Gabriel Baron³  Sally Hopewell¹  Philippe Ravaud²  Amélie Yavchitz⁴ 

[1] Centre for Statistics in Medicine, University of Oxford, Wolfson College, Oxford OX2 6UD, UK;Department of Epidemiology, Columbia University Mailman School of Public Health, New York, NY, USA;Faculté de Médecine, French Cochrane Center, Paris Descartes University, Sorbonne Paris Cité, Paris, France;Department of Anesthesiology and Critical Care, Beaujon University Hospital, Clichy, France

关键词: Results;    Abstract;    Misinterpretation;    Interpretation bias;    Interpretation;    Limitation;    Limits;    Bias;    Systematic review;    Meta-analysis;   

Others  :  1090506
DOI  :  10.1186/1471-2288-14-123

 received in 2014-07-29, accepted in 2014-11-13,  发布年份 2014

【 摘 要 】

Background

To allow an accurate evaluation of abstracts of systematic reviews, the PRISMA Statement recommends that the limitations of the evidence (e.g., risk of bias, publication bias, inconsistency, imprecision) should be described in the abstract. We aimed to evaluate the impact of adding such limitations sections on reader’s interpretation.

Method

We performed a two-arm parallel group randomized controlled trial (RCT) using a sample of 30 abstracts of systematic reviews evaluating the effects of healthcare intervention with conclusions favoring the beneficial effect of the experimental treatments. Two formats of these abstracts were derived: one reported without and one with a standardized limitations section written according to the PRISMA statement for abstracts. The primary outcome was readers’ confidence in the results of the systematic review as stated in the abstract assessed by a Likert scale from 0, not at all confident, to 10, very confident. In total, 300 participants (corresponding authors of RCT reports indexed in PubMed) were randomized by a web-based randomization procedure to interpret one abstract with a limitations section (n = 150) or without a limitations section (n = 150). Participants were blinded to the study hypothesis.

Results

Adding a limitations section did not modify readers’ interpretation of findings in terms of confidence in the results (mean difference [95% confidence interval] 0.19 [−0.37–0.74], p = 0.50), confidence in the validity of the conclusions (0.07 [−0.49–0.62], p = 0.80), or benefit of the experimental intervention (0.12 [−0.42–0.44], p = 0.65).

This study is limited because the participants were expert-readers and are not representative of all systematic review readers.

Conclusion

Adding a limitations section to abstracts of systematic reviews did not affect readers’ interpretation of the abstract results. Other studies are needed to confirm the results and explore the impact of a limitations section on a less expert panel of participants.

Trial registration

ClinicalTrial.gov (NCT01848782).

【 授权许可】

   
2014 Yavchitz et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996, 312:71-72.
• [2]Bastian H, Glasziou P, Chalmers I: Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med 2010, 7:e1000326.
• [3]Cook DJ, Mulrow CD, Haynes RB: Systematic reviews: synthesis of best evidence for clinical decisions. Ann Intern Med 1997, 126:376-380.
• [4]Bero LA, Jadad AR: How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med 1997, 127:37-42.
• [5]Shea BJ, Grimshaw JM, Wells GA, Boers M, Andersson N, Hamel C, Porter AC, Tugwell P, Moher D, Bouter LM: Development of AMSTAR: a measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007, 7:10. BioMed Central Full Text
• [6]Saint S, Christakis DA, Saha S, Elmore JG, Welsh DE, Baker P, Koepsell TD: Journal reading habits of internists. J Gen Intern Med 2000, 15:881-884.
• [7]Scherer RW, Langenberg P, von Elm E: Full publication of results initially presented in abstracts. Cochrane Database Syst Rev 2007, (2):MR000005.
• [8]The PLoS Medicine Editors: The impact of open access upon public health. PLoS Med 2006, 3:e252. doi:10.1371/journal.pmed.0030252
• [9]Hopewell S, Clarke M, Askie L: Reporting of trials presented in conference abstracts needs to be improved. J Clin Epidemiol 2006, 59:681-684.
• [10]Beller EM, Glasziou PP, Hopewell S, Altman DG: Reporting of effect direction and size in abstracts of systematic reviews. JAMA 2011, 306:1981-1982.
• [11]Liberati A, Altman DG, Tetzlaff J, Mulrow C, Gøtzsche PC, Ioannidis JPA, Clarke M, Devereaux PJ, Kleijnen J, Moher D: The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ 2009, 339:b2700.
• [12]Beller EM, Glasziou PP, Altman DG, Hopewell S, Bastian H, Chalmers I, Gøtzsche PC, Lasserson T, Tovey D: PRISMA for abstracts: reporting systematic reviews in journal and conference abstracts. PLoS Med 2013, 10:e1001419.
• [13]Hopewell S, Boutron I, Altman DG, Ravaud P: Incorporation of assessments of risk of bias of primary studies in systematic reviews of randomised trials: a cross-sectional study. BMJ Open 2013, 3:e003342.
• [14]The Editors: Addressing the limitations of structured abstracts. Ann Intern Med 2004, 140:480-481.
• [15]Schulz KF, Altman DG, Moher D: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. J Clin Epidemiol 2010, 63:834-840.
• [16]Guyatt GH: GrADe: what is “quality of evidence” and why is it important to clinicians? BMJ 2008, 336:995-998.
• [17]Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, Sch\ünemann HJ: GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336:924.
• [18]Soffer A: Abstracts of clinical investigations. A new and standardized format. Chest 1987, 92:389-390.
• [19]Huth EJ: Structured abstracts for papers reporting clinical trials. Ann Intern Med 1987, 106:626.
• [20]Hopewell S, Clarke M, Moher D, Wager E, Middleton P, Altman DG, Schulz KF: CONSORT for reporting randomized controlled trials in journal and conference abstracts: explanation and elaboration. PLoS Med 2008, 5:e20.
• [21]Hopewell S, Ravaud P, Baron G, Boutron I: Effect of editors’ implementation of CONSORT guidelines on the reporting of abstracts in high impact medical journals: interrupted time series analysis. BMJ 2012, 344:e4178.
• [22]Ioannidis JP, Lau J: Completeness of safety reporting in randomized trials: an evaluation of 7 medical areas. JAMA 2001, 285:437-443.
• [23]Yank V, Rennie D, Bero LA: Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ 2007, 335:1202-1205.
• [24]Boutron I, Dutton S, Ravaud P, Altman DG: Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA 2010, 303:2058-2064.
• [25]Kesselheim AS, Robertson CT, Myers JA, Rose SL, Gillet V, Ross KM, Glynn RJ, Joffe S, Avorn J: A randomized study of How physicians interpret research funding disclosures. N Engl J Med 2012, 367:1119-1127.
• [26]Bucher HC, Weinbacher M, Gyr K: Influence of method of reporting study results on decision of physicians to prescribe drugs to lower cholesterol concentration. BMJ 1994, 309:761-764.
• [27]Forrow L, Taylor WC, Arnold RM: Absolutely relative: how research results are summarized can affect treatment decisions. Am J Med 1992, 92:121-124.