Ensuring cross-cultural data comparability by means of anchoring vignettes in heterogeneous refugee samples

BMC Medical Research Methodology

Research

Louise Biddle¹  Natalja Menold²  Hagen von Hermanni²  Jasmin Kadel²  Kayvan Bozorgmehr³ 

[1] Charité - Universitätsmedizin Berlin, Institute of International Health, Berlin, Germany;Dept. of Methods in Empirical Social Research, Technische Universität Dresden, Dresden, Germany;University Hospital Heidelberg, Section for Health Equity Studies & Migration, Heidelberg, Germany;Dept. of Population Medicine and Health Services Research, Bielefeld University, Bielefeld, Germany;

关键词: Comparative statistics;    Measurement invariance;    Response category related differential item functioning;    Anchoring vignettes;    Health system responsiveness;    Sample heterogeneity;    Refugees;   

DOI  :  10.1186/s12874-023-02015-2

 received in 2022-05-11, accepted in 2023-08-08,  发布年份 2023

来源: Springer

【 摘 要 】

BackgroundConfigural, metric, and scalar measurement invariance have been indicators of bias-free statistical cross-group comparisons, although they are difficult to verify in the data. Low comparability of translated questionnaires or the different understanding of response formats by respondents might lead to rejection of measurement invariance and point to comparability bias in multi-language surveys. Anchoring vignettes have been proposed as a method to control for the different understanding of response categories by respondents (the latter is referred to as differential item functioning related to response categories or rating scales: RC-DIF). We evaluate the question whether the cross-cultural comparability of data can be assured by means of anchoring vignettes or by considering socio-demographic heterogeneity as an alternative approach.MethodsWe used the Health System Responsiveness (HSR) questionnaire and collected survey data in English (n = 183) and Arabic (n = 121) in a random sample of refugees in the third largest German federal state. We conducted multiple-group Confirmatory Factor Analyses (MGCFA) to analyse measurement invariance and compared the results when 1) using rescaled data on the basis of anchoring vignettes (non-parametric approach), 2) including information on RC-DIF from the analyses with anchoring vignettes as covariates (parametric approach) and 3) including socio-demographic covariates.ResultsFor the HSR, every level of measurement invariance between the Arabic and English languages was rejected. Implementing rescaling or modelling on the basis of anchoring vignettes provided superior results over the initial MGCFA analysis, since configural, metric and – for ordered categorical analyses—scalar invariance could not be rejected. A consideration of socio-demographic variables did not show such an improvement.ConclusionsSurveys may consider anchoring vignettes as a method to assess cross-cultural comparability of data, whereas socio-demographic variables cannot be used to improve data comparability as a standalone method. More research on the efficient implementation of anchoring vignettes and further development of methods to incorporate them when modelling measurement invariance is needed.

【 授权许可】

CC BY   
© BioMed Central Ltd., part of Springer Nature 2023

【 图 表 】

Fig. 5

Fig. 6

Fig. 42

13690_2023_1170_Article_IEq224.gif

Fig. 3

Fig. 3

Fig. 1

13690_2023_1170_Article_IEq220.gif

Fig. 3

12888_2023_5172_Article_IEq1.gif

Fig. 6

13690_2023_1170_Article_IEq196.gif

Fig. 3