Statistical power as a function of Cronbach alpha of instrument questionnaire items

BMC Medical Research Methodology

Myles S. Faith²  Namhee Kim³  Moonseong Heo¹ 

[1] Department of Epidemiology and Population Health, Albert Einstein College of Medicine, 1300 Morris Park Avenue, Bronx 10461, NY, USA;Department of Nutrition, Gillings School of Public Health, University of North Carolina—Chapel Hill, Chapel Hill 27599, NC, USA;Department of Radiology, Albert Einstein College of Medicine, 1300 Morris Park Avenue, Bronx 10461, NY, USA

关键词: Effect size;    Statistical power;    Reliability;    Internal consistency;    Test-retest correlation;    Coefficient alpha;    Cronbach alpha;   

Others  :  1230343
DOI  :  10.1186/s12874-015-0070-6

 received in 2015-04-18, accepted in 2015-09-18,  发布年份 2015

【 摘 要 】

Background

In countless number of clinical trials, measurements of outcomes rely on instrument questionnaire items which however often suffer measurement error problems which in turn affect statistical power of study designs. The Cronbach alpha or coefficient alpha, here denoted by C _α , can be used as a measure of internal consistency of parallel instrument items that are developed to measure a target unidimensional outcome construct. Scale score for the target construct is often represented by the sum of the item scores. However, power functions based on C _α have been lacking for various study designs.

Methods

We formulate a statistical model for parallel items to derive power functions as a function of C _α under several study designs. To this end, we assume fixed true score variance assumption as opposed to usual fixed total variance assumption. That assumption is critical and practically relevant to show that smaller measurement errors are inversely associated with higher inter-item correlations, and thus that greater C _α is associated with greater statistical power. We compare the derived theoretical statistical power with empirical power obtained through Monte Carlo simulations for the following comparisons: one-sample comparison of pre- and post-treatment mean differences, two-sample comparison of pre-post mean differences between groups, and two-sample comparison of mean differences between groups.

Results

It is shown that C _α is the same as a test-retest correlation of the scale scores of parallel items, which enables testing significance of C _α . Closed-form power functions and samples size determination formulas are derived in terms of C _α , for all of the aforementioned comparisons. Power functions are shown to be an increasing function of C _α , regardless of comparison of interest. The derived power functions are well validated by simulation studies that show that the magnitudes of theoretical power are virtually identical to those of the empirical power.

Conclusion

Regardless of research designs or settings, in order to increase statistical power, development and use of instruments with greater C _α , or equivalently with greater inter-item correlations, is crucial for trials that intend to use questionnaire items for measuring research outcomes.

Discussion

Further development of the power functions for binary or ordinal item scores and under more general item correlation strutures reflecting more real world situations would be a valuable future study.

【 授权许可】

   
2015 Heo et al.

【 图 表 】

Fig. 1.

Fig. 1.

【 参考文献 】

• [1]Hamilton M. A rating scale for depression. J Neurol Neurosurg Psychiatry. 1960; 23:56-62.
• [2]Nunnally JC, Bernstein IH. Psychometric Theory. 3rd ed. McGraw-Hill, New York; 1994.
• [3]Lord FM, Novick MR. Statistical Theories of Mental Test Scores. Addison-Wesley, Reading, MA; 1968.
• [4]Schmitt N. Uses and abuses of coefficient alpha. Psychol Assess. 1996; 8(4):350-353.
• [5]Cronbach L. Coefficeint alpha and the internal struture of tests. Psychometrika. 1951; 16:297-334.
• [6]Bland JM, Altman DG. Cronbach’s alpha. Br Med J. 1997;314(7080):572–2.
• [7]Cortina JM. What is coefficient alpha - An exmination of theory and applications. J Appl Psychol. 1993; 78(1):98-104.
• [8]Sijtsma K. On the Use, the Misuse, and the Very Limited Usefulness of Cronbach’s Alpha. Psychometrika. 2009; 74(1):107-120.
• [9]Novick MR, Lewis C. Coefficient alpha and the reliability of composite measurements. Psychometrika. 1967; 32(1):1-13.
• [10]Charter RA. Statistical approaches to achieving sufficiently high test score reliabilities for research purposes. J Gen Psychol. 2008; 135(3):241-251.
• [11]Feldt LS, Charter RA. Estimating the reliability of a test split into two parts of equal or unequal length. Psychol Methods. 2003; 8(1):102-109.
• [12]Feldt LS, Ankenmann RD. Determining sample size for a test of the equality of alpha coefficients when the number of part-tests is small. Psychol Methods. 1999; 4(4):366-377.
• [13]Feldt LS, Ankenmann RD. Appropriate sample size for comparing alpha reliabilities. Appl Psychol Meas. 1998; 22(2):170-178.
• [14]Padilla MA, Divers J, Newton M. Coefficient Alpha Bootstrap Confidence Interval Under Nonnormality. Appl Psychol Meas. 2012; 36(5):331-348.
• [15]Bonett DG, Wright TA. Cronbach’s alpha reliability: Interval estimation, hypothesis testing, and sample size planning. J Organ Behav. 2015; 36(1):3-15.
• [16]Bonett DG. Sample size requirements for testing and estimating coefficient alpha. J Educ Behav Stat. 2002; 27(4):335-340.
• [17]Bonett DG. Sample size requirements for comparing two alpha coefficients. Appl Psychol Meas. 2003; 27(1):72-74.
• [18]Donner A, Birkett N, Buck C. Randomization by cluster. Sample size requirements and analysis. Am J Epidemiol. 1981; 114(6):906-914.
• [19]Goldstein H. Multilevel Statistical Models. 2nd ed. Wiley & Sons, New York; 1996.
• [20]Leon AC, Marzuk PM, Portera L. More reliable outcome measures can reduce sample size requirements. Arch Gen Psychiatry. 1995; 52(10):867-871.
• [21]Zeger SL, Liang KY, Albert PS. Models for longitudinal data - A generalized estimating equation approach. Biometrics. 1988; 44(4):1049-1060.