How confidence intervals become confusion intervals

BMC Medical Research Methodology

G Michael Allan¹  Ben Vandermeer²  James McCormack³ 

[1] Evidence-Based Medicine, Department of Family Medicine, University of Alberta, Room 1706 College Plaza, 8215 - 112 Street NW, Edmonton AB, Canada;Alberta Research Centre for Health Evidence, University of Alberta, Edmonton Alberta, Canada;Faculty of Pharmaceutical Sciences, University of British Columbia, Vancouver BC, Canada

关键词: Statistical significance;    Statistical analysis;    Evidence based medicine;    Confidence intervals;   

Others  :  866610
DOI  :  10.1186/1471-2288-13-134

 received in 2013-08-08, accepted in 2013-10-29,  发布年份 2013

【 摘 要 】

Background

Controversies are common in medicine. Some arise when the conclusions of research publications directly contradict each other, creating uncertainty for frontline clinicians.

Discussion

In this paper, we review how researchers can look at very similar data yet have completely different conclusions based purely on an over-reliance of statistical significance and an unclear understanding of confidence intervals. The dogmatic adherence to statistical significant thresholds can lead authors to write dichotomized absolute conclusions while ignoring the broader interpretations of very consistent findings. We describe three examples of controversy around the potential benefit of a medication, a comparison between new medications, and a medication with a potential harm. The examples include the highest levels of evidence, both meta-analyses and randomized controlled trials. We will show how in each case the confidence intervals and point estimates were very similar. The only identifiable differences to account for the contrasting conclusions arise from the serendipitous finding of confidence intervals that either marginally cross or just fail to cross the line of statistical significance.

Summary

These opposing conclusions are false disagreements that create unnecessary clinical uncertainty. We provide helpful recommendations in approaching conflicting conclusions when they are associated with remarkably similar results.

【 授权许可】

   
2013 McCormack et al.; licensee BioMed Central Ltd.

【 图 表 】

【 参考文献 】

• [1]Cowles M, Davis C: On the origins of the.05 level of statistical significance. Am Psychol 1982, 37:553-558.
• [2]Studer M, Briel M, Leimenstoll B, Glass TR, Bucher HC: Effect of different antilipidemic agents and diets on mortality: A Systematic Review. Arch Intern Med 2005, 165:725-730.
• [3]Thavendiranathan P, Bagai A, Brookhart MA, Choudhry NK: Primary prevention of cardiovascular diseases with statin therapy: a meta-analysis of randomized controlled trials. Arch Intern Med 2006, 166:2307-2313.
• [4]Mills EJ, Rachlis B, Wu P, Devereaux PJ, Arora P, Perri D: Primary prevention of cardiovascular mortality and events with statin treatments: a network meta-analysis involving more than 65,000 patients. J Am Coll Cardiol 2008, 52:1769-1781.
• [5]Brugts JJ, Yetgin T, Hoeks SE, Gotto AM, Shepherd J, Westendorp RGJ, et al.: The benefits of statins in people without established cardiovascular disease but with cardiovascular risk factors: meta-analysis of randomised controlled trials. BMJ 2009, 338:b2376.
• [6]Ray KK, Seshasai SRK, Erqou S, Sever P, Jukema JW, Ford I, et al.: Statins and all-cause mortality in high-risk primary prevention: a meta-analysis of 11 randomized controlled trials involving 65,229 participants. Arch Intern Med 2010, 170:1024-1031.
• [7]Connolly SJ, Ezekowitz MD, Yusuf S, Eikelboom J, Oldgren J, Parekh A, et al.: Dabigatran versus warfarin in patients with atrial fibrillation. N Engl J Med 2009, 361:1139-1151.
• [8]Granger CB, Alexander JH, McMurray JJV, Lopes RD, Hylek EM, Hanna M, et al.: Apixaban versus warfarin in patients with atrial fibrillation. N Engl J Med 2011, 365:981-992.
• [9]Hughes S: ARISTOTLE: A major win for apixaban in AF. Heartwire; 2011. http://www.medscape.com/viewarticle/748682 webcite (accessed November 5, 2013)
• [10]Singh S, Loke YK, Spangler JG, Furberg CD: Risk of serious adverse cardiovascular events associated with varenicline: a systematic review and meta-analysis. CMAJ 2011, 183:1359-1366.
• [11]Prochaska JJ, Hilton JF: Risk of cardiovascular serious adverse events associated with varenicline use for tobacco cessation: systematic review and meta-analysis. BMJ 2012, 344:e2856.
• [12]Seshasai SR, Wijesuriya S, Sivakumaran R, Nethercott S, Erqou S, Sattar N, Ray KK: Effect of aspirin on vascular and nonvascular outcomes: meta-analysis of randomized controlled trials. Arch Intern Med 2012, 172:209-216.
• [13]Raju N, Sobieraj-Teague M, Hirsh J, O’Donnell M, Eikelboom J: Effect of aspirin on mortality in the primary prevention of cardiovascular disease. Am J Med 2011, 124:621-629.
• [14]Sterne JA, Davey Smith G: Sifting the evidence-what’s wrong with significance tests? BMJ 2001, 322:226-231.
• [15]Rothman KJ: A show of confidence. N Engl J Med 1978, 299:1362-1363.
• [16]Gardner MJ, Altman DG: Confidence intervals rather than P values: estimation rather than hypothesis testing. Br Med J (Clin Res Ed) 1986, 292:746-750.
• [17]Poole C: Beyond the confidence interval. Am J Public Health 1987, 77:195-199.
• [18]Bulpitt CJ: Confidence intervals. Lancet 1987, 329(8531):494-497.
• [19]Cooper RJ, Wears RL, Schriger DL: Reporting research results: recommendations for improving communication. Ann Emerg Med 2003, 41:561-564.
• [20]Cummings P, Koepsell TD: P values vs estimates of association with confidence intervals. Arch Pediatr Adolesc Med 2010, 164:193-196.