Measurement invariance across chronic conditions: a systematic review and an empirical investigation of the Health Education Impact Questionnaire (heiQ™)

Health and Quality of Life Outcomes

Hermann Faller¹  Richard H Osborne³  Sandra Nolte⁴  Jürgen Bengel²  Gunda Musekamp¹  Michael Schuler¹ 

[1] Department of Medical Psychology and Psychotherapy, Medical Sociology and Rehabilitation Sciences Section, University of Würzburg, Klinikstr. 3, D-97070 Wuerzburg, Germany;Department of Psychology, University of Freiburg, Engelbergerstraße 41, D-79085 Freiburg, Germany;School of Health & Social Development, Deakin University, Burwood Campus, 221 Burwood Highway, Melbourne, VIC 3125, Australia;Medical Clinic, Department of Psychosomatic Medicine, Charité – Universitätsmedizin Berlin, Charitéplatz 1, D-10117 Berlin, Germany

关键词: Systematic review;    Generic questionnaire;    Chronic disease;    Bias;    Measurement invariance;   

Others  :  814940
DOI  :  10.1186/1477-7525-12-56

 received in 2014-01-31, accepted in 2014-04-17,  发布年份 2014

【 摘 要 】

Background

To examine whether lack of measurement invariance (MI) influences mean comparisons among different disease groups, this paper provides (1) a systematic review of MI in generic constructs across chronic conditions and (2) an empirical analysis of MI in the Health Education Impact Questionnaire (heiQ™).

Methods

(1) We searched for studies of MI among different chronic conditions in online databases. (2) Multigroup confirmatory factor analyses were used to study MI among five chronic conditions (orthopedic condition, rheumatism, asthma, COPD, cancer) in the heiQ™ with N = 1404 rehabilitation inpatients. Impact on latent and composite mean differences was examined.

Results

(1) A total of 30 relevant studies suggested that about one in three items lacked MI. However, only four studies examined impact on latent mean differences. Scale means were only affected in one of these three studies. (2) Across the eight heiQ™ scales, seven scales had items with lack of MI in at least one disease group. However, in only two heiQ™ scales were some latent or composite mean differences affected.

Conclusions

Lack of MI among disease groups is common and may have a relevant influence on mean comparisons when using generic instruments. Therefore, when comparing disease groups, tests of MI should be implemented. More studies of MI and according impact on mean differences in generic questionnaires are needed.

【 授权许可】

   
2014 Schuler et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Meredith W: Measurement invariance, factor analysis and factorial invariance. Psychometrika 1993, 58:525-543.
• [2]Schmitt N, Kuljanin G: Measurement invariance: review of practice and implications. Hum Resour Manag Rev 2008, 18:210-222.
• [3]Chen FF: What happens if we compare chopsticks with forks? The impact of making inappropriate comparisons in cross-cultural research. J Pers Soc Psychol 2008, 95:1005-1018.
• [4]Millsap RE: Statistical approaches to measurement invariance. New York, NY u.a: Psychology Press; 2011.
• [5]Steinmetz H: Analyzing observed composite differences across groups is partial measurement invariance enough? Meth Eur J Res Meth Behav Soc Sci 2013, 9:1-12.
• [6]Millsap RE, Kwok O-M: Evaluating the impact of partial factorial invariance on selection in two populations. Psychol Methods 2004, 9:93-115.
• [7]Meredith W, Teresi JA: An essay on measurement and factorial invariance. Med Care 2006, 44:69-77.
• [8]Teresi JA: Overview of quantitative measurement methods. Equivalence, invariance, and differential item functioning in health applications. Med Care 2006, 44:S39-S49.
• [9]Millsap RE: Comments on methods for the investigation of measurement bias in the Mini-Mental State Examination. Med Care 2006, 44:S171-S175.
• [10]Borsboom D: When does measurement invariance matter? Med Care 2006, 44:S176-S181.
• [11]Gregorich SE: Do self-report instruments allow meaningful comparisons across diverse population groups? Testing measurement invariance using the confirmatory factor analysis framework. Med Care 2006, 44:S78-S94.
• [12]Teresi JA: Different approaches to differential item functioning in health applications. Advantages, disadvantages and some neglected topics. Med Care 2006, 44:S152-S170.
• [13]Schuler M, Jelitte M: Messen wir bei allen Personen das Gleiche? Zur Invarianz von Messungen und response shift in der rehabilitation - Teil 1. Die Rehabilitation 2012, 51:332-339.
• [14]Byrne BM, Shavelson RJ, Muthén B: Testing for the Equivalence of Factor Covariance and Mean Structures - the Issue of Partial Measurement Invariance. Psychol Bull 1989, 105:456-466.
• [15]Millsap RE, Meredith W: Factorial invariance: Historical perspectives and new problems. In Factor analysis at 100: Historical developments and future directions. Mahwah, NJ: Lawrence Erlbaum Associates Publishers; US; 2007:131-152.
• [16]Schmitt N, Golubovich J, Leong FT: Impact of measurement invariance on construct correlations, mean differences, and relations with external correlates: an illustrative example using big five and RIASEC measures. Assessment 2011, 18:412-427.
• [17]Sass DA: Testing measurement invariance and comparing latent factor means within a confirmatory factor analysis framework. J Psychoeduc Assess 2011, 29:347-363.
• [18]Stark S, Chernyshenko OS, Drasgow F: Detecting differential item functioning with confirmatory factor analysis and item response theory: toward a unified strategy. J Appl Psychol 2006, 91:1292-1306.
• [19]Teresi JA, Fleishman JA: Differential item functioning and health assessment. Qual Life Res 2007, 16(Suppl 1):33-42.
• [20]Saris WE, Satorra A, van der Veld WM: Testing structural equation models or detection of misspecifications? Struct Equ Model 2009, 16:561-582.
• [21]De Beuckelaer A, Swinnen G: Biased latent variable mean comparisons due to measurement noninvariance: A simulation study. In European Association for Methodology series European Association for Methodology series. Edited by Davidov E, Schmidt P, Billiet J. New York, NY: Routledge/Taylor & Francis Group; 2011:117-147.
• [22]Teresi JA, Ramirez M, Lai J-s, Silver S: Occurrences and sources of Differential Item Functioning (DIF) in patient-reported outcome measures: Description of DIF methods, and review of measures of depression, quality of life and general health. Psychol Sci 2008, 50:538-612.
• [23]Vandenberg RJ, Lance CE: A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organ Res Methods 2000, 3:4-69.
• [24]Bode RK, Lai JS, Cella D, Heinemann AW: Issues in the development of an item bank. Arch Phys Med Rehabil 2003, 84:S52-S60.
• [25]Dallmeijer AJ, de Groot V, Roorda LD, Schepers VP, Lindeman E, van den Berg LH, Beelen A, Dekker J: Cross-diagnostic validity of the SF-36 physical functioning scale in patients with stroke, multiple sclerosis and amyotrophic lateral sclerosis: a study using Rasch analysis. J Rehabil Med 2007, 39:163-169.
• [26]Dallmeijer AJ, Dekker J, Roorda LD, Knol DL, van Baalen B, de Groot V, Schepers VP, Lankhorst GJ: Differential item functioning of the functional independence measure in higher performing neurological patients. J Rehabil Med 2005, 37:346-352.
• [27]Lindeboom R, Holman R, Dijkgraaf MG, Sprangers MA, Buskens E, Diederiks JP, De Haan RJ: Scaling the sickness impact profile using item response theory: an exploration of linearity, adaptive use, and patient driven item weights. J Clin Epidemiol 2004, 57:66-74.
• [28]Steultjens MP, Stolwijk-Swuste J, Roorda LD, Dallmeijer AJ, van Dijk GM, Post B, Dekker J: WOMAC-pf as a measure of physical function in patients with Parkinson’s disease and late-onset sequels of poliomyelitis: unidimensionality and item behaviour. Disabil Rehabil 2012, 34:1423-1430.
• [29]Taylor WJ, McPherson KM: Using Rasch analysis to compare the psychometric properties of the Short Form 36 physical function score and the health assessment questionnaire disability index in patients with psoriatic arthritis and rheumatoid arthritis. Arthritis Rheum 2007, 57:723-729.
• [30]van Groen MM, ten Klooster PM, Taal E, van de Laar MA, Glas CA: Application of the health assessment questionnaire disability index to various rheumatic diseases. Qual Life Res 2010, 19:1255-1263.
• [31]Yu YF, Yu AP, Ahn J: Investigating differential item functioning by chronic diseases in the SF-36 health survey: a latent trait analysis using MIMIC models. Med Care 2007, 45:851-859.
• [32]Lundgren-Nilsson A, Tennant A, Grimby G, Sunnerhagen KS: Cross-diagnostic validity in a generic instrument: an example from the functional independence measure in scandinavia. Health Qual Life Out 2006, 4:55. BioMed Central Full Text
• [33]Hart DL, Mioduski JE, Stratford PW: Simulated computerized adaptive tests for measuring functional status were efficient with good discriminant validity in patients with hip, knee, or foot/ankle impairments. J Clin Epidemiol 2005, 58:629-638.
• [34]Pickard AS, Dalal MR, Bushnell DM: A comparison of depressive symptoms in stroke and primary care: applying Rasch models to evaluate the center for epidemiologic studies-depression scale. Value Health 2006, 9:59-64.
• [35]Reilly RE, Bowden SC, Bardenhagen FJ, Cook MJ: Equality of the psychological model underlying depressive symptoms in patients with temporal lobe epilepsy versus heterogeneous neurological disorders. J Clin Exp Neuropsychol 2006, 28:1257-1271.
• [36]Waller NG, Compas BE, Hollon SD, Beckjord E: Measurement of depressive symptoms in women with breast cancer and women with clinical depression: a differential item functioning analysis. J Clin Psychol Med Settings 2005, 12:127-141.
• [37]Wann-Hansson C, Klevsgard R, Hagell P: Cross-diagnostic validity of the Nottingham Health Profile Index of Distress (NHPD). Health Qual Life Out 2008, 6:47. BioMed Central Full Text
• [38]Atkinson TM, Rosenfeld BD, Sit L, Mendoza TR, Fruscione M, Lavene D, Shaw M, Li Y, Hay J, Cleeland CS, Scher HI, Breitbart WS, Basch E: Using confirmatory factor analysis to evaluate construct validity of the Brief Pain Inventory (BPI). J Pain Symptom Manag 2011, 41:558-565.
• [39]Chen CC, Bode RK: Psychometric validation of the Manual Ability Measure-36 (MAM-36) in patients with neurologic and musculoskeletal disorders. Arch Phys Med Rehabil 2010, 91:414-420.
• [40]Coster WJ, Haley SM, Andres PL, Ludlow LH, Bond TL, Ni PS: Refining the conceptual basis for rehabilitation outcome measurement: personal care and instrumental activities domain. Med Care 2004, 42:I62-I72.
• [41]Haley SM, Coster WJ, Andres PL, Ludlow LH, Ni P, Bond TL, Sinclair SJ, Jette AM: Activity outcome measurement for postacute care. Med Care 2004, 42:I49-I61.
• [42]Weisscher N, Glas CA, Vermeulen M, De Haan RJ: The use of an item response theory-based disability item bank across diseases: accounting for differential item functioning. J Clin Epidemiol 2010, 63:543-549.
• [43]Farin E, Fleitz A: The development of an ICF-oriented, adaptive physician assessment instrument of mobility, self-care, and domestic life. Int J Rehabil Res Internationale Zeitschrift fur Rehabilitationsforschung Revue internationale de recherches de readaptation 2009, 32:98-107.
• [44]Yao G, Wu CH: Factorial invariance of the WHOQOL-BREF among disease groups. Qual Life Res 2005, 14:1881-1888.
• [45]Moorer P, Suurmeije Th P, Foets M, Molenaar IW: Psychometric properties of the RAND-36 among three chronic diseases (multiple sclerosis, rheumatic diseases and COPD) in The Netherlands. Qual Life Res 2001, 10:637-645.
• [46]Yorke J, Horton M, Jones PW: A critique of Rasch analysis using the Dyspnoea-12 as an illustrative example. J Adv Nurs 2012, 68:191-198.
• [47]Roelofs J, Sluiter JK, Frings-Dresen MH, Goossens M, Thibault P, Boersma K, Vlaeyen JW: Fear of movement and (re)injury in chronic musculoskeletal pain: Evidence for an invariant two-factor model of the Tampa Scale for Kinesiophobia across pain diagnoses and Dutch, Swedish, and Canadian samples. Pain 2007, 131:181-190.
• [48]Prieto G, Delgado AR, Perea MV, Ladera V: Differential functioning of mini-mental test items according to disease. Neurologia 2011, 26:474-480.
• [49]Chien TW, Wang WC, Lin SB, Lin CY, Guo HR, Su SB: KIDMAP, a web based system for gathering patients’ feedback on their doctors. BMC Med Res Methodol 2009, 9:38. BioMed Central Full Text
• [50]Given CW, Given B, Stommel M, Collins C, King S, Franklin S: The caregiver reaction assessment (CRA) for caregivers to persons with chronic physical and mental impairments. Res Nurs Health 1992, 15:271-283.
• [51]Rao D, Choi SW, Victorson D, Bode R, Peterman A, Heinemann A, Cella D: Measuring stigma across neurological conditions: the development of the stigma scale for chronic illness (SSCI). Qual Life Res 2009, 18:585-595.
• [52]Wirtz M, Boecker M, Forkmann T, Neumann M: Evaluation of the “Consultation and Relational Empathy” (CARE) measure by means of Rasch-analysis at the example of cancer patients. Patient Educ Couns 2011, 82:298-306.
• [53]Wong ST, Nordstokke D, Gregorich S, Perez-Stable EJ: Measurement of social support across women from four ethnic groups: evidence of factorial invariance. J Cross Cult Gerontol 2010, 25:45-58.
• [54]Schuler M, Musekamp G, Faller H, Ehlebracht-König I, Gutenbrunner C, Kirchhof R, Bengel J, Nolte S, Osborne RH, Schwarze M: Assessment of proximal outcomes of self-management programs: translation and psychometric evaluation of a German version of the Health Education Impact Questionnaire (heiQ). Qual Life Res 2013, 22:1391-1403.
• [55]Osborne RH, Elsworth GR, Whitfield K: The Health Education Impact Questionnaire (heiQ): an outcomes and evaluation measure for patient education and self-management interventions for people with chronic conditions. Patient Educ Couns 2007, 66:192-201.
• [56]Crotty M, Prendergast J, Battersby MW, Rowett D, Graves SE, Leach G, Giles LC: Self-management and peer support among people with arthritis on a hospital joint replacement waiting list: a randomised controlled trial. Osteoarthr Cartil 2009, 17:1428-1433.
• [57]Francis KL, Matthews BL, Van Mechelen W, Bennell KL, Osborne RH: Effectiveness of a community-based osteoporosis education and self-management course: a wait list controlled trial. Osteoporos Int 2009, 20:1563-1570.
• [58]Nolte S, Elsworth GR, Sinclair AJ, Osborne RH: The extent and breadth of benefits from participating in chronic disease self-management courses: a national patient-reported outcomes survey. Patient Educ Couns 2007, 65:351-360.
• [59]Packer TL, Boldy D, Ghahari S, Melling L, Parsons R, Osborne RH: Self-management programs conducted within a practice setting: Who participates, who benefits and what can be learned? Patient Educ Couns 2012, 87:93-100.
• [60]Kroon FP, van der Burg LR, Buchbinder R, Osborne RH, Johnston RV, Pitt V: Self-management education programmes for osteoarthritis. Cochrane Database Syst Rev 2014, 1:CD008963.
• [61]Osborne RH, Batterham R, Livingston J: The evaluation of chronic disease self-management support across settings: the international experience of the health education impact questionnaire quality monitoring system. Nurs Clin North Am 2011, 46:255-270.
• [62]Nolte S, Elsworth GR, Sinclair AJ, Osborne RH: Tests of measurement invariance failed to support the application of the “then-test”. J Clin Epidemiol 2009, 62:1173-1180.
• [63]Sprangers MA, Schwartz CE: Integrating response shift into health-related quality of life research: a theoretical model. Soc Sci Med 1999, 48:1507-1515.
• [64]Muthén B: Latent variable modeling in heterogeneous populations. Psychometrika 1989, 54:557-585.
• [65]Muthén LK, Muthén B: Mplus User’s Guide. Muthén & Muthén: Los Angeles; 2010.
• [66]Yoon M, Millsap RE: Detecting violations of factorial invariance using data-based specification searches: A Monte Carlo study. Struct Equ Model 2007, 14:453-463.
• [67]L-t H, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model 1999, 6:1-55.
• [68]Oberski DJ: Jrule for Mplus. 091st edition. 2009. http://wiki.github.com/daob/JruleMplus/ webcite
• [69]van der Veld WM, Saris WE: Causes of generalized social trust. In European Association for Methodology series. Edited by Davidov E, Schmidt P, Billiet J. New York, NY: Routledge/Taylor & Francis Group; 2011:207-247.
• [70]Revilla MA: Measurment invariance and quality of composite scores in a face-to-face and web survey. Surv Res Methods 2013, 7:17-28.
• [71]Saris WE, Satorra A, Sörbom D: The detection and correction of specification errors in structural equation models. Sociol Methodol 1987, 17:105-129.
• [72]Hancock GR: Effect size, power, and sample size determination for structured means modeling and MIMIC approaches to between-groups hypothesis testing of means on a single latent construct. Psychometrika 2001, 66:373-388.
• [73]Cohen J: Statistical power analysis for the behavioral sciences. 2. 2print edition. Hillsdale, NJ u.a: Erlbaum; 1988.
• [74]Chen FF: Sensitivity of goodness of fit indexes to lack of measurement invariance. Struct Equ Model 2007, 14:464-504.
• [75]Meade AW: A taxonomy of effect size measures for the differential functioning of items and scales. J Appl Soc Psychol 2010, 95:728-743.
• [76]Nye CD, Drasgow F: Effect size indices for analyses of measurement equivalence: understanding the practical importance of differences between groups. J Appl Soc Psychol 2011, 96:966-980.
• [77]Muthén B, Asparouhov T: BSEM measurement invariance analysis. Mplus Webnote 17. 2013. [http://www.statmodel.com/examples/webnotes/webnote17.pdf webcite]