A tutorial on sensitivity analyses in clinical trials: the what, why, when and how

BMC Medical Research Methodology

Charles H Goldsmith⁶  George Wells⁹  Juneyoung Lee⁸  Monica Bawor³  Vincent Fruci²  Rejane Dillenburg⁴  Victoria Borg Debono⁶  Daisy Kosa⁶  Brittany Dennis⁶  Lora Giangregorio⁷  Marroon Thabane¹  Chenglin Ye⁶  Maura Marcucci⁶  Zainab Samaan⁵  Shiyuan Zhang⁶  Lawrence Mbuagbaw⁶  Lehana Thabane^1,10 

[1] GSK, Mississauga, ON, Canada;Michael G. DeGroote School of Medicine, McMaster University, Hamilton, ON, Canada;McMaster Integrative Neuroscience Discovery & Study (MiNDS) Program, McMaster University, Hamilton, ON, Canada;Department of Pediatrics, McMaster University, Hamilton, ON, Canada;Population Genomics Program, McMaster University, Hamilton, ON, Canada;Biostatistics Unit, Father Sean O’Sullivan Research Center, St Joseph’s Healthcare Hamilton, Hamilton, ON, Canada;Department of Kinesiology, University of Waterloo, Waterloo, ON, Canada;Department of Biostatistics, Korea University, Seoul, Korea;Department of Clinical Epidemiology, University of Ottawa, Ottawa, ON, Canada;Population Health Research Institute, Hamilton Health Sciences, Hamilton, ON, Canada

关键词: Robustness;    Clinical trials;    Sensitivity analysis;   

Others  :  1092283
DOI  :  10.1186/1471-2288-13-92

 received in 2012-12-11, accepted in 2013-07-10,  发布年份 2013

【 摘 要 】

Background

Sensitivity analyses play a crucial role in assessing the robustness of the findings or conclusions based on primary analyses of data in clinical trials. They are a critical way to assess the impact, effect or influence of key assumptions or variations—such as different methods of analysis, definitions of outcomes, protocol deviations, missing data, and outliers—on the overall conclusions of a study.

The current paper is the second in a series of tutorial-type manuscripts intended to discuss and clarify aspects related to key methodological issues in the design and analysis of clinical trials.

Discussion

In this paper we will provide a detailed exploration of the key aspects of sensitivity analyses including: 1) what sensitivity analyses are, why they are needed, and how often they are used in practice; 2) the different types of sensitivity analyses that one can do, with examples from the literature; 3) some frequently asked questions about sensitivity analyses; and 4) some suggestions on how to report the results of sensitivity analyses in clinical trials.

Summary

When reporting on a clinical trial, we recommend including planned or posthoc sensitivity analyses, the corresponding rationale and results along with the discussion of the consequences of these analyses on the overall findings of the study.

【 授权许可】

   
2013 Thabane et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmith CH: A tutorial on pilot studies: the what, why and how. BMC Med Res Methodol 2010, 10:1. BioMed Central Full Text
• [2]Schneeweiss S: Sensitivity analysis and external adjustment for unmeasured confounders in epidemiologic database studies of therapeutics. Pharmacoepidemiol Drug Saf 2006, 15(5):291-303.
• [3]Viel JF, Pobel D, Carre A: Incidence of leukaemia in young people around the La Hague nuclear waste reprocessing plant: a sensitivity analysis. Stat Med 1995, 14(21–22):2459-2472.
• [4]Goldsmith CH, Gafni A, Drummond MF, Torrance GW, Stoddart GL: Sensitivity Analysis and Experimental Design: The Case of Economic Evaluation of Health Care Programmes. Proceedings of the Third Canadian Conference on Health Economics 1986. Winnipeg MB: The University of Manitoba Press; 1987.
• [5]Saltelli A, Tarantola S, Campolongo F, Ratto M: Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models. New York, NY: Willey; 2004.
• [6]Saltelli A, Ratto M, Andres T, Campolongo F, Cariboni J, Gatelli D, Saisana M, Tarantola S: Global Sensitivity Analysis: The Primer. New York, NY: Wiley-Interscience; 2008.
• [7]Hunink MGM, Glasziou PP, Siegel JE, Weeks JC, Pliskin JS, Elstein AS, Weinstein MC: Decision Making in Health and Medicine: Integrating Evidence and Values.. Cambridge: Cambridge University Press; 2001.
• [8]USFDA: International Conference on Harmonisation; Guidance on Statistical Principles for Clinical Trials. Guideline E9. Statistical principles for clinical trials. Federal Register, 16 September 1998, Vol. 63, No. 179, p. 49583 . [http://www.fda.gov/downloads/RegulatoryInformation/Guidances/UCM129505.pdf webcite]
• [9]NICE: Guide to the methods of technology appraisal . [http://www.nice.org.uk/media/b52/a7/tamethodsguideupdatedjune2008.pdf webcite]
• [10]Ma J, Thabane L, Kaczorowski J, Chambers L, Dolovich L, Karwalajtys T, Levitt C: Comparison of Bayesian and classical methods in the analysis of cluster randomized controlled trials with a binary outcome: the Community Hypertension Assessment Trial (CHAT). BMC Med Res Methodol 2009, 9:37. BioMed Central Full Text
• [11]Peters TJ, Richards SH, Bankhead CR, Ades AE, Sterne JA: Comparison of methods for analysing cluster randomized trials: an example involving a factorial design. Int J Epidemiol 2003, 32(5):840-846.
• [12]Chu R, Thabane L, Ma J, Holbrook A, Pullenayegum E, Devereaux PJ: Comparing methods to estimate treatment effects on a continuous outcome in multicentre randomized controlled trials: a simulation study. BMC Med Res Methodol 2011, 11:21. BioMed Central Full Text
• [13]Kleinbaum DG, Klein M: Survival Analysis – A-Self Learning Text. 3rd edition. Springer; 2012.
• [14]Barnett V, Lewis T: Outliers in Statistical Data. 3rd edition. John Wiley & Sons; 1994.
• [15]Grubbs FE: Procedures for detecting outlying observations in samples. Technometrics 1969, 11:1-21.
• [16]Thabane L, Akhtar-Danesh N: Guidelines for reporting descriptive statistics in health research. Nurse Res 2008, 15(2):72-81.
• [17]Williams NH, Edwards RT, Linck P, Muntz R, Hibbs R, Wilkinson C, Russell I, Russell D, Hounsome B: Cost-utility analysis of osteopathy in primary care: results from a pragmatic randomized controlled trial. Fam Pract 2004, 21(6):643-650.
• [18]Zetta S, Smith K, Jones M, Allcoat P, Sullivan F: Evaluating the Angina Plan in Patients Admitted to Hospital with Angina: A Randomized Controlled Trial. Cardiovascular Therapeutics 2011, 29(2):112-124.
• [19]Morden JP, Lambert PC, Latimer N, Abrams KR, Wailoo AJ: Assessing methods for dealing with treatment switching in randomised controlled trials: a simulation study. BMC Med Res Methodol 2011, 11:4. BioMed Central Full Text
• [20]White IR, Walker S, Babiker AG, Darbyshire JH: Impact of treatment changes on the interpretation of the Concorde trial. AIDS 1997, 11(8):999-1006.
• [21]Borrelli B: The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent 2011, 71(Suppl 1):S52-S63.
• [22]Lawton J, Jenkins N, Darbyshire JL, Holman RR, Farmer AJ, Hallowell N: Challenges of maintaining research protocol fidelity in a clinical care setting: a qualitative study of the experiences and views of patients and staff participating in a randomized controlled trial. Trials 2011, 12:108. BioMed Central Full Text
• [23]Ye C, Giangregorio L, Holbrook A, Pullenayegum E, Goldsmith CH, Thabane L: Data withdrawal in randomized controlled trials: Defining the problem and proposing solutions: a commentary. Contemp Clin Trials 2011, 32(3):318-322.
• [24]Horwitz RI, Horwitz SM: Adherence to treatment and health outcomes. Arch Intern Med 1993, 153(16):1863-1868.
• [25]Peduzzi P, Wittes J, Detre K, Holford T: Analysis as-randomized and the problem of non-adherence: an example from the Veterans Affairs Randomized Trial of Coronary Artery Bypass Surgery. Stat Med 1993, 12(13):1185-1195.
• [26]Montori VM, Guyatt GH: Intention-to-treat principle. CMAJ 2001, 165(10):1339-1341.
• [27]Gibaldi M, Sullivan S: Intention-to-treat analysis in randomized trials: who gets counted? J Clin Pharmacol 1997, 37(8):667-672.
• [28]Porta M: A dictionary of epidemiology. 5th edition. Oxford: Oxford University Press, Inc; 2008.
• [29]Everitt B: Medical statistics from A to Z. 2nd edition. Cambridge: Cambridge University Press; 2006.
• [30]Sainani KL: Making sense of intention-to-treat. PM R 2010, 2(3):209-213.
• [31]Bendtsen P, McCambridge J, Bendtsen M, Karlsson N, Nilsen P: Effectiveness of a proactive mail-based alcohol internet intervention for university students: dismantling the assessment and feedback components in a randomized controlled trial. J Med Internet Res 2012, 14(5):e142.
• [32]Brox JI, Nygaard OP, Holm I, Keller A, Ingebrigtsen T, Reikeras O: Four-year follow-up of surgical versus non-surgical therapy for chronic low back pain. Ann Rheum Dis 2010, 69(9):1643-1648.
• [33]McKnight PE, McKnight KM, Sidani S, Figueredo AJ: Missing Data: A Gentle Introduction. New York, NY: Guilford; 2007.
• [34]Graham JW: Missing data analysis: making it work in the real world. Annu Rev Psychol 2009, 60:549-576.
• [35]Little RJ, D'Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, et al.: The Prevention and Treatment of Missing Data in Clinical Trials. New England Journal of Medicine 2012, 367(14):1355-1360.
• [36]Little RJA, Rubin DB: Statistical Analysis with Missing Data. 2nd edition. New York NY: Wiley; 2002.
• [37]Rubin DB: Multiple Imputation for Nonresponse in Surveys. John Wiley & Sons, Inc: New York NY; 1987.
• [38]Schafer JL: Analysis of Incomplete Multivariate Data. New York: Chapman and Hall; 1997.
• [39]Son H, Friedmann E, Thomas SA: Application of pattern mixture models to address missing data in longitudinal data analysis using SPSS. Nursing research 2012, 61(3):195-203.
• [40]Peters SA, Bots ML, den Ruijter HM, Palmer MK, Grobbee DE, Crouse JR 3rd, O'Leary DH, Evans GW, Raichlen JS, Moons KG, et al.: Multiple imputation of missing repeated outcome measurements did not add to linear mixed-effects models. J Clin Epidemiol 2012, 65(6):686-695.
• [41]Zhang H, Paik MC: Handling missing responses in generalized linear mixed model without specifying missing mechanism. J Biopharm Stat 2009, 19(6):1001-1017.
• [42]Chen HY, Gao S: Estimation of average treatment effect with incompletely observed longitudinal data: application to a smoking cessation study. Statistics in medicine 2009, 28(19):2451-2472.
• [43]Ma J, Akhtar-Danesh N, Dolovich L, Thabane L: Imputation strategies for missing binary outcomes in cluster randomized trials. BMC Med Res Methodol 2011, 11:18. BioMed Central Full Text
• [44]Kingsley GH, Kowalczyk A, Taylor H, Ibrahim F, Packham JC, McHugh NJ, Mulherin DM, Kitas GD, Chakravarty K, Tom BD, et al.: A randomized placebo-controlled trial of methotrexate in psoriatic arthritis. Rheumatology (Oxford) 2012, 51(8):1368-1377.
• [45]de Pauw BE, Sable CA, Walsh TJ, Lupinacci RJ, Bourque MR, Wise BA, Nguyen BY, DiNubile MJ, Teppler H: Impact of alternate definitions of fever resolution on the composite endpoint in clinical trials of empirical antifungal therapy for neutropenic patients with persistent fever: analysis of results from the Caspofungin Empirical Therapy Study. Transpl Infect Dis 2006, 8(1):31-37.
• [46]A randomized, double-blind, futility clinical trial of creatine and minocycline in early Parkinson disease Neurology 2006, 66(5)):664-671.
• [47]Song P-K: Correlated Data Analysis: Modeling, Analytics and Applications. New York, NY: Springer Verlag; 2007.
• [48]Pintilie M: Competing Risks: A Practical Perspective. New York, NY: John Wiley; 2006.
• [49]Tai BC, Grundy R, Machin D: On the importance of accounting for competing risks in pediatric brain cancer: II. Regression modeling and sample size. Int J Radiat Oncol Biol Phys 2011, 79(4):1139-1146.
• [50]Holbrook JT, Wise RA, Gold BD, Blake K, Brown ED, Castro M, Dozor AJ, Lima JJ, Mastronarde JG, Sockrider MM, et al.: Lansoprazole for children with poorly controlled asthma: a randomized controlled trial. JAMA 2012, 307(4):373-381.
• [51]Holbrook A, Thabane L, Keshavjee K, Dolovich L, Bernstein B, Chan D, Troyan S, Foster G, Gerstein H: Individualized electronic decision support and reminders to improve diabetes care in the community: COMPETE II randomized trial. CMAJ: Canadian Medical Association journal = journal de l’Association medicale canadienne 2009, 181(1–2):37-44.
• [52]Hilbe JM: Negative Binomial Regression. 2nd edition. Cambridge: Cambridge University Press; 2011.
• [53]Forsblom C, Harjutsalo V, Thorn LM, Waden J, Tolonen N, Saraheimo M, Gordin D, Moran JL, Thomas MC, Groop PH: Competing-risk analysis of ESRD and death among patients with type 1 diabetes and macroalbuminuria. J Am Soc Nephrol 2011, 22(3):537-544.
• [54]Grams ME, Coresh J, Segev DL, Kucirka LM, Tighiouart H, Sarnak MJ: Vascular disease, ESRD, and death: interpreting competing risk analyses. Clin J Am Soc Nephrol 2012, 7(10):1606-1614.
• [55]Lim HJ, Zhang X, Dyck R, Osgood N: Methods of competing risks analysis of end-stage renal disease and mortality among people with diabetes. BMC Med Res Methodol 2010, 10:97. BioMed Central Full Text
• [56]Chu R, Walter SD, Guyatt G, Devereaux PJ, Walsh M, Thorlund K, Thabane L: Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome–a simulation study. PLoS One 2012, 7(5):e36677.
• [57]Bowen A, Hesketh A, Patchick E, Young A, Davies L, Vail A, Long AF, Watkins C, Wilkinson M, Pearl G, et al.: Effectiveness of enhanced communication therapy in the first four months after stroke for aphasia and dysarthria: a randomised controlled trial. BMJ 2012, 345:e4407.
• [58]Spiegelhalter DJ, Best NG, Lunn D, Thomas A: Bayesian Analysis using BUGS: A Practical Introduction. New York, NY: Chapman and Hall; 2009.
• [59]Byers AL, Allore H, Gill TM, Peduzzi PN: Application of negative binomial modeling for discrete outcomes: a case study in aging research. J Clin Epidemiol 2003, 56(6):559-564.
• [60]Yusuf S, Wittes J, Probstfield J, Tyroler HA: Analysis and interpretation of treatment effects in subgroups of patients in randomized clinical trials. JAMA: the journal of the American Medical Association 1991, 266(1):93-98.
• [61]Altman DG: Better reporting of randomised controlled trials: the CONSORT statement. BMJ 1996, 313(7057):570-571.
• [62]Mauskopf JA, Sullivan SD, Annemans L, Caro J, Mullins CD, Nuijten M, Orlewska E, Watkins J, Trueman P: Principles of good practice for budget impact analysis: report of the ISPOR Task Force on good research practices–budget impact analysis. Value Health 2007, 10(5):336-347.