Evaluation of a weighting approach for performing sensitivity analysis after multiple imputation

BMC Medical Research Methodology

Julie A. Simpson³  John B. Carlin²  Katherine J. Lee²  Ian R. White¹  Panteha Hayati Rezvan³ 

[1]MRC Biostatistics Unit, Cambridge Institute of Public Health, Cambridge CB2 0SR, UK

[2]Department of Paediatrics, The University of Melbourne, Parkville, Melbourne, VIC, Australia

[3]Centre for Epidemiology and Biostatistics, Melbourne School of Population and Global Health, The University of Melbourne, Parkville, Melbourne, VIC, Australia

关键词: Sensitivity analysis;    Weighting approach;    Missing not at random;    Selection model;    Multiple imputation;   

Others  :  1228727
DOI  :  10.1186/s12874-015-0074-2

 received in 2015-06-03, accepted in 2015-09-28,  发布年份 2015

【 摘 要 】

Background

Multiple imputation (MI) is a well-recognised statistical technique for handling missing data. As usually implemented in standard statistical software, MI assumes that data are ‘Missing at random’ (MAR); an assumption that in many settings is implausible. It is not possible to distinguish whether data are MAR or ‘Missing not at random’ (MNAR) using the observed data, so it is desirable to discover the impact of departures from the MAR assumption on the MI results by conducting sensitivity analyses. A weighting approach based on a selection model has been proposed for performing MNAR analyses to assess the robustness of results obtained under standard MI to departures from MAR.

Methods

In this article, we use simulation to evaluate the weighting approach as a method for exploring possible departures from MAR, with missingness in a single variable, where the parameters of interest are the marginal mean (and probability) of a partially observed outcome variable and a measure of association between the outcome and a fully observed exposure. The simulation studies compare the weighting-based MNAR estimates for various numbers of imputations in small and large samples, for moderate to large magnitudes of departure from MAR, where the degree of departure from MAR was assumed known. Further, we evaluated a proposed graphical method, which uses the dataset with missing data, for obtaining a plausible range of values for the parameter that quantifies the magnitude of departure from MAR.

Results

Our simulation studies confirm that the weighting approach outperformed the MAR approach, but it still suffered from bias. In particular, our findings demonstrate that the weighting approach provides biased parameter estimates, even when a large number of imputations is performed. In the examples presented, the graphical approach for selecting a range of values for the possible departures from MAR did not capture the true parameter value of departure used in generating the data.

Conclusions

Overall, the weighting approach is not recommended for sensitivity analyses following MI, and further research is required to develop more appropriate methods to perform such sensitivity analyses.

【 授权许可】

   
2015 Hayati Rezvan et al.

【 图 表 】

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

Fig. 5.

【 参考文献 】

• [1]Hayati Rezvan P, Lee KJ, Simpson JA. The rise of multiple imputation: a review of the reporting and implementation of the method in medical research. BMC Med Res Methodol. 2015; 15(1):1-14. BioMed Central Full Text
• [2]Bell ML, Fiero M, Horton NJ, Chiu-Hsieh H. Handling missing data in RCTs; a review of the top medical journals. BMC Med Res Methodol. 2014; 14(1):1-16. BioMed Central Full Text
• [3]Powney M, Williamson P, Kirkham J, Kolamunnage-Dona R. A review of the handling of missing longitudinal outcome data in clinical trials. Trials. 2014; 15(1):1-19. BioMed Central Full Text
• [4]Karahalios A, Baglietto L, Carlin JB, English DR, Simpson JA. A review of the reporting and handling of missing data in cohort studies with repeated assessment of exposure measures. BMC Med Res Methodol. 2012; 12:96-105. BioMed Central Full Text
• [5]Wood AM, White IR, Thompson SG. Are missing outcome data adequately handled? A review of published randomized controlled trials in major medical journals. Clin Trials. 2004; 1(4):368-376.
• [6]Carpenter JR, Kenward MG. Multiple imputation and its application/James R. Carpenter and Michael G. Kenward. 1st ed. Wiley, Chichester; 2013.
• [7]Little RJA, Rubin DB. Statistical analysis with missing data/Roderick J.A. Little, Donald B. Rubin. 2nd ed. Wiley, Hoboken; 2002.
• [8]Schafer JL, Graham JW. Missing data: Our view of the state of the art. Psychol Methods. 2002; 7(2):147-177.
• [9]R Development Core Team. R: A language and environment for statistical computing, reference index version 2.2.1. Vienna, Austria. ISBN 3-900051-07-0. URL http://www.R-project.org.: R Foundation for Statistical Computing; 2005.
• [10]PROC MI. SAS Procedures Giude, Version 9.2. SAS Institute Inc, Cary; 2008.
• [11]Stata Statistical Software: Release 12. College Station, TX. Stata Corp LP, College Station, TX; 2009.
• [12]Mackinnon A. The use and reporting of multiple imputation in medical research - a review. J Intern Med. 2010; 268(6):586-593.
• [13]Sterne JAC, White IR, Carlin JB, Spratt M, Royston P, Kenward MG et al.. Multiple imputation for missing data in epidemiological and clinical research: Potential and pitfalls. BMJ (Online). 2009; 339(7713):157-160.
• [14]Kenward MG, Carpenter J. Multiple imputation: current perspectives. Stat Methods Med Res. 2007; 16(3):199-218.
• [15]Rubin DB. Multiple imputation for nonresponse in surveys/Donald B. Rubin. Wiley, New York; 1987.
• [16]Schafer JL. Analysis of incomplete multivariate data. 1st ed. Chapman & Hall/CRC, Boca Raton; 1997.
• [17]White IR, Carlin JB. Bias and efficiency of multiple imputation compared with complete-case analysis for missing covariate values. Stat Med. 2010; 29(28):2920-2931.
• [18]Carpenter JR, Kenward MG, White IR. Sensitivity analysis after multiple imputation under missing at random: a weighting approach. Stat Methods Med Res. 2007; 16(3):259-275.
• [19]O'Kelly M, Ratitch B. Clinical trials with missing data : a guide for practitioners / Michael O’Kelly, Bohdana Ratitch. John Wiley & Sons, Chichester; 2014.
• [20]Ratitch B, O'Kelly M, Tosiello R. Missing data in clinical trials: from clinical assumptions to statistical analysis using pattern mixture models. Pharm Stat. 2013; 12(6):337-347.
• [21]van Buuren S, Boshuizen HC, Knook DL. Multiple imputation of missing blood pressure covariates in survival analysis. Stat Med. 1999; 18(6):681-694.
• [22]Kenward M, Molenberghs G. Parametric models for incomplete continuous and categorical longitudinal data. Stat Methods Med Res. 1999; 8(1):51-83.
• [23]Hogan JW, Laird NM. Model-Based Approaches To Analysing Incomplete Longitudinal And Failure Time Data. Stat Med. 1997; 16(3):259-272.
• [24]Little RJA. Modeling the drop-out mechanism in repeated-measures studies. J Am Stat Assoc. 1995; 90(431):1112-1121.
• [25]Diggle P, Kenward MG. Informative Drop-out in Longitudinal Data Analysis. J R Stat Soc: Ser C: Appl Stat. 1994; 43(1):49-93.
• [26]Little RJA. Pattern-Mixture Models for Multivariate Incomplete Data. J Am Stat Assoc. 1993; 88(421):125-134.
• [27]Yuan Y. Sensitivity Analysis in Multiple Imputation for Missing Data. In Proceedings of the SAS Global Forum 2014 Conference: [http://support.sas.com/resources/papers/proceedings14/SAS270-2014.pdf].
• [28]Resseguier N, Giorgi R, Paoletti X. Sensitivity analysis when data are missing not-at-random. Epidemiology. 2011; 22(2):282-283.
• [29]Daniels MJ, Hogan JW. Missing data in longitudinal studies : strategies for Bayesian modeling and sensitivity analysis / Michael J. Daniels, Joseph W. Hogan. Chapman & Hall/CRC, Boca Raton; 2008.
• [30]Héraud-Bousquet V, Larsen C, Carpenter J, Desenclos J, Le Strat Y. Practical considerations for sensitivity analysis after multiple imputation applied to epidemiological studies with incomplete data. BMC Med Res Methodol. 2012; 12:73-83. BioMed Central Full Text
• [31]Rasbah J. A user’s guide to MLwiN, version 2.10: Centre for Multilevel Modelling. University of Bristol, Bristol, UK; 2009.
• [32]Gilks WR, Richardson S, Spiegelhalter DJ. Markov chain Monte Carlo in practice. Chapman & Hall, London; Melbourne; 1996.
• [33]Molenberghs G, Beunkens C, Jansen I, Thijs H, van Steen K, Verbeke G et al.. Analysis of incomplete data. In: Pharmaceutical statistics using SAS : a practical guide. Dmitrienko A, Chuang-Stein C, D'Agostino RB, editors. SAS publishing, Cary, NC; 2007: p.313.
• [34]Kenward MG. Selection models for repeated measurements with non-random dropout: an illustration of sensitivity. Stat Med. 1998; 17(23):2723-2732.
• [35]White IR, Carpenter J, Evans S, Schroter S. Eliciting and using expert opinions about dropout bias in randomized controlled trials. Clin Trials. 2007; 4(2):125-139.
• [36]O'Hagan A. Eliciting Expert Beliefs in Substantial Practical Applications. J R Stat Soc Series D. 1998; 47(1):21-35.
• [37]Kadane JB, Wolfson LJ. Experiences in Elicitation. J R Stat Soc Series D. 1998; 47(1):3-19.
• [38]Gelman A, Carlin JB, Stern HS, Dunson DB, Vehtari A, Rubin DB. Bayesian data analysis. 3rd ed. CRC Press, Boca Raton; 2014.
• [39]Hesterberg T. Weighted average importance sampling and defensive mixture distributions. Technometrics. 1995; 37(2):185-194.
• [40]Agresti A. An introduction to categorical data analysis/Alan Agresti. 2nd ed. Wiley-Interscience, Hoboken, NJ; 2007.
• [41]van Buuren S. Flexible Imputation of Missing Data. 1st ed. Taylor and Francis, Hoboken; 2012.
• [42]White IR, Royston P, Wood AM. Multiple imputation using chained equations: Issues and guidance for practice. Stat Med. 2011; 30(4):377-399.
• [43]Royston P. Multiple imputation of missing values. STATA J. 2004; 4(3):227-241.
• [44]White IR, Daniel R, Royston P. Avoiding bias due to perfect prediction in multiple imputation of incomplete categorical variables. Comput Stat Data Anal. 2010; 54:2267-2275.
• [45]Carpenter J, Rücker G, Schwarzer G. Assessing the Sensitivity of Meta-analysis to Selection Bias: A Multiple Imputation Approach. Biometrics. 2011; 67(3):1066-1072.
• [46]Bond SJ, White IR. Estimating causal effects using prior information on nontrial treatments. Clin Trials. 2010; 7(6):664-676.
• [47]Turner RM, Spiegelhalter DJ, Smith GCS, Thompson SG. Bias modelling in evidence synthesis. J R Stat Soc Series A. 2009; 172(1):21-47.
• [48]Siddique J, Harel O, Crespi CM. Addressing Missing Data Mechanism Uncertainty using Multiple-Model Multiple Imputation: Application to a Longitudinal Clinical Trial. Ann Appl Stat. 2012; 6(4):1814-1837.
• [49]Siddique J, Harel O, Crespi CM, Hedeker D. Binary variable multiple-model multiple imputation to address missing data mechanism uncertainty: application to a smoking cessation trial. Stat Med. 2014; 33(17):3013-3028.