Comparison of registered and published outcomes in randomized controlled trials: a systematic review

BMC Medicine

Timothy F. Platts-Mills³  Melissa C. Caughey⁴  Wesley C. Holland⁵  Lukas G. Keil¹  Christopher W. Jones² 

[1]School of Medicine, University of North Carolina Chapel Hill, 321 S Columbia St, Chapel Hill 27516, NC, USA

[2]Department of Emergency Medicine, Cooper Medical School of Rowan University, One Cooper Plaza, Suite 152, Camden 08103, NJ, USA

[3]Department of Emergency Medicine, University of North Carolina Chapel Hill, 170 Manning Dr. CB#7594, Chapel Hill 27599, NC, USA

[4]Department of Medicine, University of North Carolina Chapel Hill, 125 MacNider Hall, CB 7005, Chapel Hill 27599, NC, USA

[5]Department of Biology, University of North Carolina Chapel Hill, Coker Hall, 120 South Rd, Chapel Hill 27599, NC, USA

关键词: Trial registration;    Selection bias;    Reporting bias;    Publication bias;    Primary outcome;    Clinicaltrials.gov;   

Others  :  1233461
DOI  :  10.1186/s12916-015-0520-3

 received in 2015-08-11, accepted in 2015-11-03,  发布年份 2015

【 摘 要 】

Background

Clinical trial registries can improve the validity of trial results by facilitating comparisons between prospectively planned and reported outcomes. Previous reports on the frequency of planned and reported outcome inconsistencies have reported widely discrepant results. It is unknown whether these discrepancies are due to differences between the included trials, or to methodological differences between studies. We aimed to systematically review the prevalence and nature of discrepancies between registered and published outcomes among clinical trials.

Methods

We searched MEDLINE via PubMed, EMBASE, and CINAHL, and checked references of included publications to identify studies that compared trial outcomes as documented in a publicly accessible clinical trials registry with published trial outcomes. Two authors independently selected eligible studies and performed data extraction. We present summary data rather than pooled analyses owing to methodological heterogeneity among the included studies.

Results

Twenty-seven studies were eligible for inclusion. The overall risk of bias among included studies was moderate to high. These studies assessed outcome agreement for a median of 65 individual trials (interquartile range [IQR] 25–110). The median proportion of trials with an identified discrepancy between the registered and published primary outcome was 31 %; substantial variability in the prevalence of these primary outcome discrepancies was observed among the included studies (range 0 % (0/66) to 100 % (1/1), IQR 17–45 %). We found less variability within the subset of studies that assessed the agreement between prospectively registered outcomes and published outcomes, among which the median observed discrepancy rate was 41 % (range 30 % (13/43) to 100 % (1/1), IQR 33–48 %). The nature of observed primary outcome discrepancies also varied substantially between included studies. Among the studies providing detailed descriptions of these outcome discrepancies, a median of 13 % of trials introduced a new, unregistered outcome in the published manuscript (IQR 5–16 %).

Conclusions

Discrepancies between registered and published outcomes of clinical trials are common regardless of funding mechanism or the journals in which they are published. Consistent reporting of prospectively defined outcomes and consistent utilization of registry data during the peer review process may improve the validity of clinical trial publications.

【 授权许可】

   
2015 Jones et al.

【 图 表 】

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

【 参考文献 】

• [1]Moher D, Hopewell S, Schulz KF, Montori V, Gotzsche PC, Devereaux PJ et al.. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010; 340:c869.
• [2]Newman DH, Shreves AE. Treatment of acute otitis media in children. N Engl J Med. 2011; 364(18):1775.
• [3]DeAngelis CD, Drazen JM, Frizelle FA, Haug C, Hoey J, Horton R et al.. Clinical trial registration: a statement from the International Committee of Medical Journal Editors. JAMA. 2004; 292(11):1363-4.
• [4]Food and Drug Administration Act of 2007, Stat. 2007: 110–85.
• [5]Regulation (EU) No 536/2014 of the European Parliament and of the Council of 16 April 2014 on clinical trials on medicinal products for human use, and repealing Directive 2001/20/ED, Stat. 536/2014. 2014.
• [6]Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. Characteristics of clinical trials registered in ClinicalTrials.gov, 2007–2010. JAMA. 2012; 307(17):1838-47.
• [7]Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomized controlled trials. JAMA. 2009; 302(9):977-84.
• [8]Ross JS, Mulvey GK, Hines EM, Nissen SE, Krumholz HM. Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis. PLoS Med. 2009; 6(9):e1000144.
• [9]Smith SM, Wang AT, Pereira A, Chang RD, McKeown A, Greene K et al.. Discrepancies between registered and published primary outcome specifications in analgesic trials: ACTTION systematic review and recommendations. Pain. 2013; 154(12):2769-74.
• [10]Chan AW, Hrobjartsson A, Haahr MT, Gotzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA. 2004; 291(20):2457-65.
• [11]Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. BMJ. 2009; 339:b2535.
• [12]Higgins J, Green S, (editors). Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011]. London, UK: The Cochrane Collaboration; 2011.
• [13]Wells GA, Shea B, O’Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analyses. Ottawa Hospital Research Institute. 2014. http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp. Accessed 24 September 2015.
• [14]Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998; 17(8):857-72.
• [15]Ewart R, Lausen H, Millian N. Undisclosed changes in outcomes in randomized controlled trials: an observational study. Ann Fam Med. 2009; 7(6):542-6.
• [16]Anand V, Scales DC, Parshuram CS, Kavanagh BP. Registration and design alterations of clinical trials in critical care: a cross-sectional observational study. Intensive Care Med. 2014; 40(5):700-22.
• [17]Bourgeois FT, Murthy S, Mandl KD. Outcome reporting among drug trials registered in ClinicalTrials.gov. Ann Intern Med. 2010; 153(3):158-66.
• [18]Chahal J, Tomescu SS, Ravi B, Bach BR, Ogilvie-Harris D, Mohamed NN et al.. Publication of sports medicine-related randomized controlled trials registered in ClinicalTrials.gov. Am J Sports Med. 2012; 40(9):1970-7.
• [19]Gandhi R, Jan M, Smith HN, Mahomed NN, Bhandari M. Comparison of published orthopaedic trauma trials following registration in Clinicaltrials.gov. BMC Musculoskelet Disord. 2011; 12:278. BioMed Central Full Text
• [20]Hannink G, Gooszen HG, Rovers MM. Comparison of registered and published primary outcomes in randomized clinical trials of surgical interventions. Ann Surg. 2013; 257(5):818-23.
• [21]Hartung DM, Zarin DA, Guise JM, McDonagh M, Paynter R, Helfand M. Reporting discrepancies between the ClinicalTrials.gov results database and peer-reviewed publications. Ann Intern Med. 2014; 160(7):477-83.
• [22]Huic M, Marusic M, Marusic A. Completeness and changes in registered data and reporting bias of randomized controlled trials in ICMJE journals after trial registration policy. PLoS One. 2011; 6(9):e25258.
• [23]Hutfless S, Blair R, Berger Z, Wilson L, Bass EB, Lazarev M. Poor reporting of Crohn’s disease trials in ClinicalTrials.gov. Gastroenterology. 2013; 144(5):SUPPL. 1(S232).
• [24]Jones CW, Platts-Mills TF. Quality of registration for clinical trials published in emergency medicine journals. Ann Emerg Med. 2012; 60(4):458-64.
• [25]Khan NA, Lombeida JI, Singh M, Spencer HJ, Torralba KD. Association of industry funding with the outcome and quality of randomized controlled trials of drug therapy for rheumatoid arthritis. Arthritis Rheum. 2012; 64(7):2059-67.
• [26]Killeen S, Sourallous P, Hunter IA, Hartley JE, Grady HL. Registration rates, adequacy of registration, and a comparison of registered and published primary outcomes in randomized controlled trials published in surgery journals. Ann Surg. 2014; 259(1):193-6.
• [27]Li XQ, Yang GL, Tao KM, Zhang HQ, Zhou QH, Ling CQ. Comparison of registered and published primary outcomes in randomized controlled trials of gastroenterology and hepatology. Scand J Gastroenterol. 2013; 48(12):1474-83.
• [28]Liu JP, Han M, Li XX, Mu YJ, Lewith G, Wang YY, et al. Prospective registration, bias risk and outcome-reporting bias in randomised clinical trials of traditional Chinese medicine: an empirical methodological study. BMJ Open. 2013;3(7):pii: e002968. doi:10.1136/bmjopen-2013-002968.
• [29]Mathieu S, Giraudeau B, Soubrier M, Ravaud P. Misleading abstract conclusions in randomized controlled trials in rheumatology: comparison of the abstract conclusions and the results section. Joint Bone Spine. 2012; 79(3):262-7.
• [30]Milette K, Roseman M, Thombs BD. Transparency of outcome reporting and trial registration of randomized controlled trials in top psychosomatic and behavioral health journals: a systematic review. J Psychosom Res. 2011; 70(3):205-17.
• [31]Nankervis H, Baibergenova A, Williams HC, Thomas KS. Prospective registration and outcome-reporting bias in randomized controlled trials of eczema treatments: a systematic review. J Invest Dermatol. 2012; 132(12):2727-34.
• [32]Pinto RZ, Elkins MR, Moseley AM, Sherrington C, Herbert RD, Maher CG et al.. Many randomized trials of physical therapy interventions are not adequately registered: a survey of 200 published trials. Phys Ther. 2013; 93(3):299-309.
• [33]Rosenthal R, Dwan K. Comparison of randomized controlled trial registry entries and content of reports in surgery journals. Ann Surg. 2013; 257(6):1007-15.
• [34]Smith HN, Bhandari M, Mahomed NN, Jan M, Gandhi R. Comparison of arthroplasty trial publications after registration in ClinicalTrials.gov. J Arthroplasty. 2012; 27(7):1283-8.
• [35]Vera-Badillo FE, Shapiro R, Ocana A, Amir E, Tannock IF. Bias in reporting of end points of efficacy and toxicity in randomized, clinical trials for women with breast cancer. Ann Oncol. 2013; 24(5):1238-44.
• [36]Walker KF, Stevenson G, Thornton JG. Discrepancies between registration and publication of randomised controlled trials: an observational study. JRSM Open. 2014; 5(5):2042533313517688.
• [37]Wildt S, Krag A, Gluud L. Characteristics of randomised trials on diseases in the digestive system registered in ClinicalTrials.gov: a retrospective analysis. BMJ Open. 2011; 1(2):e000309.
• [38]You B, Gan HK, Pond G, Chen EX. Consistency in the analysis and reporting of primary end points in oncology randomized controlled trials from registration to publication: a systematic review. J Clin Oncol. 2012; 30(2):210-6.
• [39]Mathieu S, Chan AW, Ravaud P. Use of trial register information during the peer review process. PLoS One. 2013; 8(4):e59910.
• [40]Chauvin A, Ravaud P, Baron G, Barnes C, Boutron I. The most important tasks for peer reviewers evaluating a randomized controlled trial are not congruent with the tasks most often requested by journal editors. BMC Med. 2015; 13:158. BioMed Central Full Text
• [41]Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011; 364(9):852-60.
• [42]Dwan K, Altman DG, Cresswell L, Blundell M, Gamble CL, Williamson PR. Comparison of protocols and registry entries to published reports for randomised controlled trials. Cochrane Database Syst Rev. 2011; 1:MR000031.