Shame for disrespecting evidence: the personal consequences of insufficient respect for structural equation model testing

BMC Medical Research Methodology

Leslie A Hayduk¹ 

[1] Department of Sociology, University of Alberta, Edmonton, Canada

关键词: SEM;    Structural equation model;    Close fit;    Testing;    Factor model;    Factor analysis;   

Others  :  1090486
DOI  :  10.1186/1471-2288-14-124

 received in 2014-05-15, accepted in 2014-11-13,  发布年份 2014

【 摘 要 】

Background

Inappropriate and unacceptable disregard for structural equation model (SEM) testing can be traced back to: factor-analytic inattention to model testing, misapplication of the Wilkinson task force’s [Am Psychol 54:594-604, 1999] critique of tests, exaggeration of test biases, and uncomfortably-numerous model failures.

Discussion

The arguments for disregarding structural equation model testing are reviewed and found to be misguided or flawed. The fundamental test-supporting observations are: a) that the null hypothesis of the χ² structural equation model test is not nil, but notable because it contains substantive theory claims and consequences; and b) that the amount of covariance ill fit cannot be trusted to report the seriousness of model misspecifications. All covariance-based fit indices risk failing to expose model problems because the extent of model misspecification does not reliably correspond to the magnitude of covariance ill fit – seriously causally misspecified models can fit, or almost fit.

Summary

The only reasonable research response to evidence of non-chance structural equation model failure is to diagnostically investigate the reasons for failure. Unfortunately, many SEM-based theories and measurement scales will require reassessment if we are to clear the backlogged consequences of previous deficient model testing. Fortunately, it will be easier for researchers to respect evidence pointing toward required reassessments, than to suffer manuscript rejection and shame for disrespecting evidence potentially signaling serious model misspecifications.

【 授权许可】

   
2014 Hayduk; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Rodgers JL: The epistemology of mathematical and statistical modeling: A quiet methodological revolution. Am Psychol 2010, 65:1-12.
• [2]Wilkinson L, The Task Force on Statistical Inference: Statistical Methods in Psychology Journals: Guidelines and Explanations. Am Psychol 1999, 54:594-604.
• [3]SEMNET: The Structural Equation Modeling Discussion Network. http://www.aime.ua.edu/cgi-bin/wa?A0=SEMNET webcite
• [4]Joreskog K, Sorbom D: LISREL III: Estimation of Linear Structural Equation Systems by Maximum Likelihood Methods, User’s Guide. Chicago: National Educational Resources, Inc.; 1976.
• [5]Entwisle DR, Hayduk LA, Reilly TW: Early Schooling: Cognitive and Affective Outcomes. Baltimore: Johns Hopkins University Press; 1982.
• [6]Sorbom D: Karl Joreskog and LISREL: A personal story. In Structural Equation Modeling: Present and Future. A Festschrift in Honor of Karl Joreskog. Edited by Cudeck R, du Toit S, Sorbom D. Lincolnwood, IL: Scientific Software International; 2001:3-10.
• [7]Hayduk LA, Glaser DN: Jiving the four-step, waltzing around factor analysis, and other serious fun. Struct Equ Model 2000, 7:1-35.
• [8]Barrett P: Structural equation modelling: Adjudging model fit. Person Individ Differ 2007, 42:815-824.
• [9]Hayduk LA, Cummings GG, Boadu K, Pazderka-Robinson H, Boulianne S: Testing! Testing! One, two, three – Testing the theory in structural equation models! Person Individ Differ 2007, 42:841-850.
• [10]Bollen KA: Structural Equations with Latent Variables. New York: Wiley-Interscience; 1989.
• [11]Hayduk LA: Structural Equation Modeling with LISREL: Essentials and Advances. Baltimore: Johns Hopkins University Press; 1987.
• [12]Satorra A, Bentler PM: Ensuring positiveness of the scaled difference chi-square test statistic. Psychometrika 2010, 75:243-248.
• [13]Ledermann W: On the rank of the reduced correlation matrix in multiple-factor analysis. Psychometrica 1937, 2:85-93.
• [14]Hayduk LA: Seeing perfectly fitting factor models that are causally misspecified: Understanding that close-fitting models can be worse. Educ Psychol Meas 2014, 74:905-926. doi:10.1177/0013164414527449
• [15]Savalei V: The relationship between root mean square error of approximation and model misspecification in confirmatory factor analysis models. Educ Psychol Meas 2012, 72(6):910-932.
• [16]Heene M, Hilbert S, Freudenthaler HH, Buhner M: Sensitivity of SEM fit indexes with respect to violations of uncorrelated errors. Struct Equ Model 2012, 19:36-50.
• [17]Browne MW, Cudeck R: Alternative ways of assessing model fit. Soc Method Res 1992, 21(2):230-258.
• [18]Hu L, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model 1999, 6:1-55.
• [19]Box GEP: Robustness in the strategy of scientific model building. In Robustness in Statistics. Edited by Launer RL, Wilkinson GN. New York: Academic Press; 1979:201-236.
• [20]Box GEP, Draper NR: Empirical Model-Building and Response Surfaces. New York: John Wiley & Sons; 1987.
• [21]Box GEP: Some problems of statistics and everyday life. J Amer Stat Assoc 1979, 74:1-4.
• [22]Joreskog K, Sorbom D: LISREL 8 User’s Reference Guide. Chicago, IL: Scientific Software International, Inc.; 1996.
• [23]Fan X, Thompson B, Wang L: Effects of sample size, estimation methods, and model specification on structural equation modeling fit indexes. Struct Equ Model 1999, 6:56-83.
• [24]Box GEP, Hill WJ: Discriminating among mechanistic models. Technometrics 1967, 9:57-71.
• [25]Box GEP: Science and statistics. J Amer Stat Assoc 1976, 71:791-799.
• [26]Benjamini Y, Hochberg Y: Controlling the false discovery rate: A practical and powerful approach to multiple testing. J Royal Stat Soc 1995, 57(Series B):289-300.
• [27]Herting JR, Costner HL: Another perspective on “the proper number of factors” and the appropriate number of steps. Struct Equ Model 2000, 7:92-110.
• [28]Satorra A, Bentler PM: Corrections to test statistics and standard errors in covariance structure analysis. In Latent Variable Analysis in Developmental Research. Edited by von Eye A, Clogg C. Newbury Park, Calif: Sage Publications; 1994:285-305.
• [29]Muthen BO: Goodness of fit with categorical and other nonnormal variables. In Testing Structural Equation Models. Edited by Bollen KA, Long JS. Newbury Park, Calif: Sage Publications; 1993:205-234.
• [30]Curran PJ, Bollen KA, Paxton P, Kirby J, Chen F: The noncentral chi-square distribution in misspecified structural equation models: Finite sample results from a Monte Carlo simulation. Multivar Behav Res 2002, 37:1-36.
• [31]Browne MW, MacCallum RC, Kim CT, Andersen BL, Glaser R: When fit indices and residuals are incompatible. Psychol Meth 2002, 7:403-421.
• [32]Hayduk LA, Pazderka-Robinson H, Cummings GG, Levers MD, Beres MA: Structural equation model testing and the quality of natural killer cell activity measurements. BMC Med Res Methodol 2005., 5(1) doi:10.1186/1471-2288-5-1
• [33]Hayduk LA: Personal Space: Understanding the Simplex Model. J Nonverbal Behav 1994, 18(3):245-260.
• [34]Hayduk LA, Stratkotter RF, Rovers MW: Sexual orientation and the willingness of Catholic seminary students to conform to church teachings. J Sci Stud Relig 1997, 36(3):455-467.
• [35]Hayduk LA, Pazderka-Robinson H, Cummings GG, Boadu K, Verbeek EL, Perks TA: The weird world, and equally weird measurement models: Reactive indicators and the validity revolution. Struct Equ Model 2007, 14(2):280-310.
• [36]Hayduk LA, Littvay L: Should researchers use single indicators, best indicators, or multiple indicators in structural equation models. BMC Med Res Methodol 2012, 12:159. doi:10.1186/1471-2288-12-159 BioMed Central Full Text
• [37]Hayduk LA, Pazderka-Robinson H: Fighting to understand the world causally: Three battles connected to the causal implications of structural equation models. In Sage Handbook of Social Science Methodology. Edited by Outhwaite W, Turner S. London: Sage Publications; 2007:147-171.