Bias, precision and statistical power of analysis of covariance in the analysis of randomized trials with baseline imbalance: a simulation study

BMC Medical Research Methodology

Julius Sim¹  Martyn Lewis¹  Bolaji E Egbewale¹ 

[1] Research Institute for Primary Care and Health Sciences, Keele University, ST5 5BG Staffordshire, UK

关键词: Statistical power;    Precision;    Bias;    Baseline imbalance;    Randomized controlled trials;    Statistical analysis;   

Others  :  866363
DOI  :  10.1186/1471-2288-14-49

 received in 2013-11-30, accepted in 2014-03-31,  发布年份 2014

【 摘 要 】

Background

Analysis of variance (ANOVA), change-score analysis (CSA) and analysis of covariance (ANCOVA) respond differently to baseline imbalance in randomized controlled trials. However, no empirical studies appear to have quantified the differential bias and precision of estimates derived from these methods of analysis, and their relative statistical power, in relation to combinations of levels of key trial characteristics. This simulation study therefore examined the relative bias, precision and statistical power of these three analyses using simulated trial data.

Methods

126 hypothetical trial scenarios were evaluated (126 000 datasets), each with continuous data simulated by using a combination of levels of: treatment effect; pretest-posttest correlation; direction and magnitude of baseline imbalance. The bias, precision and power of each method of analysis were calculated for each scenario.

Results

Compared to the unbiased estimates produced by ANCOVA, both ANOVA and CSA are subject to bias, in relation to pretest-posttest correlation and the direction of baseline imbalance. Additionally, ANOVA and CSA are less precise than ANCOVA, especially when pretest-posttest correlation ≥ 0.3. When groups are balanced at baseline, ANCOVA is at least as powerful as the other analyses. Apparently greater power of ANOVA and CSA at certain imbalances is achieved in respect of a biased treatment effect.

Conclusions

Across a range of correlations between pre- and post-treatment scores and at varying levels and direction of baseline imbalance, ANCOVA remains the optimum statistical method for the analysis of continuous outcomes in RCTs, in terms of bias, precision and statistical power.

【 授权许可】

   
2014 Egbewale et al.; licensee BioMed Central Ltd.

【 图 表 】

【 参考文献 】

• [1]Rosenberger WF, Lachin JM: Randomization in Clinical Trials: Theory and Practice. New York, NY: Wiley-Interscience; 2002.
• [2]Roberts C, Torgerson DJ: Baseline imbalance in randomised controlled trials. BMJ 1999, 319:185.
• [3]Altman DG, Doré CJ: Baseline comparisons in randomized clinical trials. Stat Med 1991, 10:797-799.
• [4]Tu D, Shalay K, Pater J: Adjustment of treatment effect for covariates in clinical trials: statistical and regulatory issues. Drug Inf J 2000, 34:511-523.
• [5]Ciolino JD, Martin RH, Zhao W, Jauch EC, Hill MD, Palesch YY: Covariate imbalance and adjustment for logistic regression analysis of clinical trial data. J Biopharm Stat 2013, 23:1383-1402.
• [6]Piantadosi S: Clinical Trials: a Methodologic Perspective. 2nd edition. New York: Wiley; 2005.
• [7]Kernan WN, Makuch RM: Response. J Clin Epidemiol 2001, 54:105.
• [8]Scott NW, McPherson GC, Ramsay CR, Campbell MK: The method of minimization for allocation to clinical trials: a review. Control Clin Trials 2002, 23:662-674.
• [9]Hagino A, Hamada C, Yoshimura I, Ohashi Y, Sakamoto J, Nakazato H: Statistical comparison of random allocation methods in cancer clinical trials. Control Clin Trials 2004, 25:572-584.
• [10]Taves DR: Faulty assumptions in Atkinson’s criteria for clinical trial design. J R Stat Soc 2004, 167:179-181.
• [11]Rosenberger WF, Sverdlov O: Handling covariates in the design of clinical trials. Stat Sci 2008, 23:404-419.
• [12]Frison L, Pocock SJ: Repeated measures in clinical trials: analysis using mean summary statistics and its implications for design. Stat Med 1992, 11:1685-1704.
• [13]Hernández AV, Eijkemans MJ, Steyerberg EW: Randomized controlled trials with time-to-event outcomes: how much does prespecified covariate adjustment increase power? Ann Epidemiol 2006, 16:41-48.
• [14]Van Breukelen GJP: ANCOVA versus change from baseline had more power in randomized studies and more bias in nonrandomized studies. J Clin Epidemiol 2006, 59:920-925.
• [15]Kent DM, Trikalinos TA, Hill MD: Are unadjusted analyses of clinical trials inappropriately biased toward the null? Stroke 2009, 40:672-673.
• [16]Ciolino JD, Martin RH, Zhao W, Hill MD, Jauch EC, Palesch YY: Measuring continuous baseline covariate imbalances in clinical trial data. Stat Methods Med Res 2011. doi:10.1177/0962280211416038
• [17]Austin PC, Manca A, Zwarenstein M, Juurlink DN, Stanbrook MB: A substantial and confusing variation exists in handling of baseline covariates in randomized controlled trials: a review of trials published in leading medical journals. J Clin Epidemiol 2010, 63:142-153.
• [18]Vickers AJ: The use of percentage change from baseline as an outcome in a controlled trial is statistically inefficient: a simulation study. BMC Med Res Methodol 2001, 1:6. BioMed Central Full Text
• [19]Matthews JNS: Introduction to Randomized Controlled Clinical Trials. 2nd edition. Boca Raton, FL: Chapman & Hall/CRC; 2006.
• [20]Huitema B: The Analysis of Covariance and Alternatives: Statistical Methods for Experiments, Quasi-Experiments, and Single-Case Studies. 2nd edition. Hoboken, NJ: Wiley; 2011.
• [21]Christensen E, Neuberger J, Crowe J, Altman DG, Popper H, Portmann B, Doniach D, Ranek L, Tygstrup N, Williams R: Beneficial effect of azathioprine and prediction of prognosis in primary biliary cirrhosis. Final results of an international trial. Gastroenterology 1985, 89:1084-1091.
• [22]Beach ML, Meier P: Choosing covariates in the analysis of clinical trials. Control Clin Trials 1989, 10(4 Suppl):161S-175S.
• [23]Steyerberg EW, Bossuyt PMM, Lee KL: Clinical trials in acute myocardial infarction: should we adjust for baseline characteristics? Am Heart J 2000, 139:745-751.
• [24]Vickers AJ, Altman DG: Analysing controlled trials with baseline and follow up measurements. BMJ 2001, 323:1123-1124.
• [25]Senn SJ: Baseline comparisons in randomised clinical trials. Stat Med 1991, 10:1157-1160.
• [26]Overall JE, Magee KN: Directional baseline differences and Type I error probabilities in randomized clinical trials. J Biopharm Stat 1992, 2:189-203.
• [27]Overall JE, Doyle SR: Implications of chance baseline differences in repeated measurement designs. J Biopharm Stat 1994, 4:199-216.
• [28]Chu R, Walter SD, Guyatt G, Devereaux PJ, Walsh M, Thorlund K, Thabane L: Assessment and implication of prognostic imbalance in randomized controlled trials with a binary outcome – a simulation study. PLoS One 2012, 7:e36677.
• [29]Pocock SJ, Assmann SE, Enos LE, Kasten LE: Subgroup analysis, covariate adjustment and baseline comparisons in clinical trial reporting: current practice and problems. Stat Med 2002, 21:2917-2930.
• [30]Cohen J: Statistical Power Analysis for the Behavioral Sciences. 2nd edition. Hillsdale NJ: Lawrence Erlbaum; 1988.
• [31]Tu YK, Blance A, Clerehugh V, Gilthorpe MS: Statistical power for analyses of changes in randomized controlled trials. J Dent Res 2005, 84:283-287.
• [32]Wei L, Zhang J: Analysis of data with imbalance in the baseline outcome variable for randomized clinical trials. Drug Inf J 2001, 35:1201-1214.
• [33]Egger MJ, Coleman ML, Ward JR, Reading JC, Williams HJ: Uses and abuses of analysis of covariance in clinical trials. Control Clin Trials 1985, 6:12-24.
• [34]Twisk J, Proper K: Evaluation of the result of a randomized controlled trial: how to define changes between baseline and follow up. J Clin Epidemiol 2004, 57:223-228.
• [35]Assmann SF, Pocock SJ, Enos LE, Kasten LE: Subgroup analysis and other (mis)uses of baseline data in clinical trials. Lancet 2000, 355:1064-1069.
• [36]Armitage P, Gehan EA: Statistical methods for the identification and use of prognostic factors. Int J Cancer 1974, 13:16-36.
• [37]Senn SJ: Covariate imbalance and random allocation in clinical trials. Stat Med 1989, 8:467-475.
• [38]Senn SJ: Testing for baseline balance in clinical trials. Stat Med 1994, 13:1715-1726.
• [39]Raab GM, Day S, Sales J: How to select covariates to include in the analysis of a clinical trial. Control Clin Trials 2000, 21:330-342.
• [40]DHHS: Guidance for Industry E9: Statistical Principles for Clinical Trials. Rockville MD: Department of Health and Human Services; 1998.
• [41]Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hróbjartsson A, Schulz KF, Parulekar WR, Krleža-Jeric K, Laupacis A, Moher D: SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ 2013, 346:e7586.
• [42]Ciolino J, Zhao W, Martin R, Palesch Y: Quantifying the cost in power of ignoring continuous covariates imbalances in clinical trial randomization. Contemp Clin Trials 2011, 32:250-259.
• [43]Kernan WN, Viscoli CM, Makuch RW, Brass LM, Horwitz RI: Stratified randomization for clinical trials. J Clin Epidemiol 1999, 52:19-26.
• [44]Kahan BC, Morris TP: Reporting and analysis of trials using stratified randomisation in leading medical journals: review and reanalysis. BMJ 2012, 345:e5840.
• [45]Overall JE, Ashby B: Baseline corrections in experimental and quasi-experimental clinical trials. Neuropsychopharmacology 1991, 4:273-281.
• [46]Hernández AV, Steyerberg EW, Habbema JD: Covariate adjustment in randomized controlled trials with dichotomous outcomes increases statistical power and reduces sample size requirements. J Clin Epidemiol 2004, 57:454-460.