Simulation methods to estimate design power: an overview for applied research

BMC Medical Research Methodology

Alan E Hubbard²  John M Colford³  Daniel R Hogan¹  Benjamin F Arnold³ 

[1] Center for Health Decision Science, Harvard School of Public Health, Boston, MA, USA;Division of Biostatistics, School of Public Health, University of California, Berkeley, CA, USA;Division of Epidemiology, School of Public Health, University of California, Berkeley, CA, USA

关键词: Sample Size;    Research Design;    Power;    Computer Simulation;   

Others  :  1140151
DOI  :  10.1186/1471-2288-11-94

 received in 2011-02-01, accepted in 2011-06-20,  发布年份 2011

【 摘 要 】

Background

Estimating the required sample size and statistical power for a study is an integral part of study design. For standard designs, power equations provide an efficient solution to the problem, but they are unavailable for many complex study designs that arise in practice. For such complex study designs, computer simulation is a useful alternative for estimating study power. Although this approach is well known among statisticians, in our experience many epidemiologists and social scientists are unfamiliar with the technique. This article aims to address this knowledge gap.

Methods

We review an approach to estimate study power for individual- or cluster-randomized designs using computer simulation. This flexible approach arises naturally from the model used to derive conventional power equations, but extends those methods to accommodate arbitrarily complex designs. The method is universally applicable to a broad range of designs and outcomes, and we present the material in a way that is approachable for quantitative, applied researchers. We illustrate the method using two examples (one simple, one complex) based on sanitation and nutritional interventions to improve child growth.

Results

We first show how simulation reproduces conventional power estimates for simple randomized designs over a broad range of sample scenarios to familiarize the reader with the approach. We then demonstrate how to extend the simulation approach to more complex designs. Finally, we discuss extensions to the examples in the article, and provide computer code to efficiently run the example simulations in both R and Stata.

Conclusions

Simulation methods offer a flexible option to estimate statistical power for standard and non-traditional study designs and parameters of interest. The approach we have described is universally applicable for evaluating study designs used in epidemiologic and social science research.

【 授权许可】

   
2011 Arnold et al; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

【 参考文献 】

• [1]Murray DM: Design and Analysis of Group-Randomized Trials. Oxford University Press; 1998.
• [2]Heo M, Leon AC: Statistical power and sample size requirements for three level hierarchical cluster randomized trials. Biometrics 2008, 64:1256-1262.
• [3]Roy A, Bhaumik DK, Aryal S, Gibbons RD: Sample size determination for hierarchical longitudinal designs with differential attrition rates. Biometrics 2007, 63:699-707.
• [4]Pocock SJ, Simon R: Sequential treatment assignment with balancing for prognostic factors in the controlled clinical trial. Biometrics 1975, 31:103-115.
• [5]Pocock SJ: Group sequential methods in the design and analysis of clinical trials. Biometrika 1977, 64:191-199.
• [6]Abbas I, Rovira J, Casanovas J, Greenfield T: Optimal design of clinical trials with computer simulation based on results of earlier trials, illustrated with a lipodystrophy trial in HIV patients. J Biomed Inform 2008, 41:1053-1061.
• [7]Gastanaga VM, McLaren CE, Delfino RJ: Power calculations for generalized linear models in observational longitudinal studies: a simulation approach in SAS. Comput Methods Programs Biomed 2006, 84:27-33.
• [8]Taylor DW, Bosch EG: CTS: A clinical trials simulator. Stat Med 1990, 9:787-801.
• [9]Eng J: Sample size estimation: a glimpse beyond simple formulas. Radiology 2004, 230:606-612.
• [10]Guimaraes P, Palesch Y: Power and sample size simulations for Randomized Play-the-Winner rules. Contemp Clin Trials 2007, 28:487-499.
• [11]Sutton AJ, Cooper NJ, Jones DR, Lambert PC, Thompson JR, Abrams KR: Evidence-based sample size calculations based upon updated meta-analysis. Stat Med 2007, 26:2479-2500.
• [12]Reynolds R, Lambert PC, Burton PR, on Resistance Surveillance BSACEWP: Analysis, power and design of antimicrobial resistance surveillance studies, taking account of inter-centre variation and turnover. J Antimicrob Chemother 2008, 62(Suppl 2):ii29-ii39.
• [13]Sutton AJ, Donegan S, Takwoingi Y, Garner P, Gamble C, Donald A: An encouraging assessment of methods to inform priorities for updating systematic reviews. J Clin Epidemiol 2009, 62:241-251.
• [14]Orloff J, Douglas F, Pinheiro J, Levinson S, Branson M, Chaturvedi P, Ette E, Gallo P, Hirsch G, Mehta C, Patel N, Sabir S, Springs S, Stanski D, Evers MR, Fleming E, Singh N, Tramontin T, Golub H: The future of drug development: advancing clinical trial design. Nat Rev Drug Discov 2009, 8:949-957.
• [15]Moineddin R, Matheson FI, Glazier RH: A simulation study of sample size for multilevel logistic regression models. BMC Med Res Methodol 2007, 7:34. BioMed Central Full Text
• [16]Feiveson AH: Power by simulation. Stata Journal 2002, 2:107-124.
• [17]Shumway-Cook A, Silver IF, LeMier M, York S, Cummings P, Koepsell TD: Effectiveness of a community-based multifactorial intervention on falls and fall risk factors in community-living older adults: a randomized, controlled trial. J Gerontol A Biol Sci Med Sci 2007, 62:1420-1427.
• [18]Baqui AH, El-Arifeen S, Darmstadt GL, Ahmed S, Williams EK, Seraji HR, Mannan I, Rahman SM, Shah R, Saha SK, Syed U, Winch PJ, Lefevre A, Santosham M, Black RE, Group PS: Effect of community-based newborn-care intervention package implemented through two service-delivery strategies in Sylhet district, Bangladesh: a cluster-randomised controlled trial. Lancet 2008, 371:1936-1944.
• [19]Victora CG, Adair L, Fall C, Hallal PC, Martorell R, Richter L, Sachdev HS, Group CUS: Maternal and child undernutrition: consequences for adult health and human capital. Lancet 2008, 371:340-357.
• [20]Checkley W, Epstein LD, Gilman RH, Black RE, Cabrera L, Sterling CR: Effects of Cryptosporidium parvum infection in Peruvian children: growth faltering and subsequent catch-up growth. Am J Epidemiol 1998, 148:497-506.
• [21]Checkley W, Buckley G, Gilman RH, Assis AM, Guerrant RL, Morris SS, Mølbak K, Valentiner-Branth P, Lanata CF, Black RE, Malnutrition C, Network I: Multi-country analysis of the effects of diarrhoea on childhood stunting. Int J Epidemiol 2008, 37:816-830.
• [22]Esrey SA, Potash JB, Roberts L, Shiff C: Effects of improved water supply and sanitation on ascariasis, diarrhoea, dracunculiasis, hookworm infection, schistosomiasis, and trachoma. Bull World Health Organ 1991, 69:609-621.
• [23]Waddington H, Snilstveit B: Effectiveness and sustainability of water, sanitation, and hygiene interventions in combating diarrhoea. J Dev Eff 2009, 1:295-335.
• [24]Clasen TF, Bostoen K, Schmidt W-P, Boisson S, Fung IC-H, Jenkins MW, Scott B, Sugden S, Cairncross S: Interventions to improve disposal of human excreta for preventing diarrhoea. Cochrane Database Syst Rev 2010., 6
• [25]Esrey SA: Water, waste, and well-being: a multicountry study. Am J Epidemiol 1996, 143:608-623.
• [26]Checkley W, Gilman RH, Black RE, Epstein LD, Cabrera L, Sterling CR, Moulton LH: Effect of water and sanitation on childhood health in a poor Peruvian peri-urban community. Lancet 2004, 363:112-118.
• [27]Bhutta ZA, Ahmed T, Black RE, Cousens S, Dewey K, Giugliani E, Haider BA, Kirkwood B, Morris SS, Sachdev HPS, Shekar M, Group CUS: What works? Interventions for maternal and child undernutrition and survival. Lancet 2008, 371:417-440.
• [28]Lunn PG: The impact of infection and nutrition on gut function and growth in childhood. Proc Nutr Soc 2000, 59:147-154.
• [29]Humphrey JH: Child undernutrition, tropical enteropathy, toilets, and handwashing. Lancet 2009, 374:1032-1035.
• [30]Kar K: Subsidy or self-respect? Participatory total community sanitation in Bangladesh. IDS Working Paper 184 2003.
• [31]Adu-Afarwuah S, Lartey A, Brown KH, Zlotkin S, Briend A, Dewey KG: Randomized comparison of 3 types of micronutrient supplements for home fortification of complementary foods in Ghana: effects on growth and motor development. Am J Clin Nutr 2007, 86:412-420.
• [32]Adu-Afarwuah S, Lartey A, Brown KH, Zlotkin S, Briend A, Dewey KG: Home fortification of complementary foods with micronutrient supplements is well accepted and has positive effects on infant iron status in Ghana. Am J Clin Nutr 2008, 87:929-938.
• [33]Phuka JC, Maleta K, Thakwalakwa C, Cheung YB, Briend A, Manary MJ, Ashorn P: Complementary feeding with fortified spread and incidence of severe stunting in 6- to 18-month-old rural Malawians. Arch Pediatr Adolesc Med 2008, 162:619-626.
• [34]Phuka JC, Maleta K, Thakwalakwa C, Cheung YB, Briend A, Manary MJ, Ashorn P: Postintervention growth of Malawian children who received 12-mo dietary complementation with a lipid-based nutrient supplement or maize-soy flour. Am J Clin Nutr 2009, 89:382-390.
• [35]Hayes RJ, Bennett S: Simple sample size calculation for cluster-randomized trials. Int J Epidemiol 1999, 28:319-326.
• [36]Laird NM, Ware JH: Random-effects models for longitudinal data. Biometrics 1982, 38:963-974.
• [37]Freedman DA: On the so-called "Huber sandwich estimator" and "robust standard errors.". Am Stat 2006, 60:299-302.
• [38]Zeger SL, Liang KY: Longitudinal data analysis for discrete and continuous outcomes. Biometrics 1986, 42:121-130.
• [39]Hubbard AE, Ahern J, Fleischer NL, der Laan MV, Lippman SA, Jewell N, Bruckner T, Satariano WA: To GEE or not to GEE: Comparing population average and mixed models for estimating the associations between neighborhood risk factors and health. Epidemiology 2010, 21:467-474.
• [40]Abadie A: Bootstrap tests for distributional treatment effects in instrumental variable models. J Am Stat Assoc 2002, 97:284-292.
• [41]Sekhon JS: Multivariate and propensity score matching software with automated balance optimization: The Matching package for R. J Stat Softw 2010, in press.
• [42]Cameron L, Shaw M: Scaling Up Rural Sanitation: Findings from the Impact Evaluation Baseline Survey in Indonesia. [http:/ / www.wsp.org/ wsp/ sites/ wsp.org/ files/ publications/ WSP_IndonesiaBaselineReport_TSSM.pd f] webciteWater and Sanitation Program Technical Paper The World Bank; 2010.
• [43]WHO: WHO Child Growth Standards: Length/height-for-age, weight-for-age, weight-for-length, weight-for-height and body mass index-for-age: Methods and developments. 2006.
• [44]Arnold BF, Khush RS, Ramaswamy P, London AG, Rajkumar P, Ramaprabha P, Durairaj N, Hubbard AE, Balakrishnan K, Colford JM: Causal inference methods to study nonrandomized, preexisting development interventions. Proc Natl Acad Sci USA 2010, 107:22605-22610.
• [45]Hubbard AE, van der Laan MJ: Population intervention models in causal inference. Biometrika 2008, 95:35-47.
• [46]Fleischer NL, Fernald LCH, Hubbard AE: Estimating the potential impacts of intervention from observational data: methods for estimating causal attributable risk in a cross-sectional analysis of depressive symptoms in Latin America. J Epidemiol Community Health 2010, 64:16-21.
• [47]Schulz KF, Altman DG, Moher D, Group CONSORT: CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. PLoS Med 2010., 7
• [48]Burton A, Altman DG, Royston P, Holder RL: The design of simulation studies in medical statistics. Stat Med 2006, 25:4279-4292.