Mode differences in a mixed-mode health interview survey among adults

Archives of Public Health

Jens Hoebel¹  Elena von der Lippe¹  Cornelia Lange¹  Thomas Ziese¹ 

[1] Department of Epidemiology and Health Monitoring, Robert Koch Institute, General-Pape-Straße 62-66, 12101 Berlin, Germany

关键词: Health indicators;    Mode effects;    Survey methods;    Mixed-mode;    Public health surveillance;    Data collection;    Health surveys;   

Others  :  1128600
DOI  :  10.1186/2049-3258-72-46

 received in 2014-05-14, accepted in 2014-09-01,  发布年份 2014

【 摘 要 】

Background

Health interview surveys are important data sources for empirical research in public health. However, the diversity of methods applied, such as in the mode of data collection, make it difficult to compare results across surveys, time, or countries. The aim of this study was to explore whether the prevalence rates of health-related indicators amongst adults differ when self-administered paper mail questionnaires (SAQ-Paper), self-administered web surveys (SAQ-Web), and computer-assisted telephone interviews (CATI) are used for data collection in a health survey.

Methods

Data were obtained from a population-based mixed-mode health interview survey of adults in Germany carried out within the ‘German Health Update’ (GEDA) study. Data were collected either by SAQ-Paper (n = 746), SAQ-Web (n = 414), or CATI (n = 411). Predictive margins from logistic regression models were used to estimate the prevalence rates of chronic conditions, subjective health, mental health, psychosocial factors, and health behaviours, adjusted for the socio-demographic characteristics of each mode group.

Results

Socio-demographic characteristics were found to differ significantly between study participants who responded by SAQ-Paper, SAQ-Web, and CATI. Crude prevalence rates for health-related indicators also showed significant variation across all three survey modes. After adjusting for socio-demographic factors though, significant differences in prevalence rates between the two self-administered modes (SAQ-Paper and SAQ-Web) were found in only 2 out of the 19 health-related indicators studied. The differences between CATI and the two self-administered modes remained significant however, especially for indicators of mental and psychosocial health and self-reported sporting activity.

Conclusions

The findings of this study indicate that prevalence rates obtained from health interview surveys can vary with the mode of data collection, primarily between interviewer and self-administered modes. Hence, the type of survey mode used should be considered when comparing results from different health surveys. Mixing self-administered modes, such as paper-based questionnaires and web surveys, may be a combination to minimize mode differences in mixed-mode health interview surveys.

【 授权许可】

   
2014 Hoebel et al.; licensee BioMed Central.

【 参考文献 】

• [1]Koponen P, Aromaa A: Survey Design and Methodology in National Health Interview and Health Examination Surveys. Review of Literature, European Survey Experiences and Recommendations. Helsinki: National Public Health Institute (KTL); 2000.
• [2]De Leeuw ED: To mix or not to mix data collection modes in surveys. J Off Stat 2005, 21:233-255.
• [3]Dillman D, Smyth JD, Christian LM: Internet, mail, and misex-node surveys: the tailored design method. Hoboken, New Jersey: John Wiley & Sons, Inc.; 2009.
• [4]Krosnick JA: Response strategies for coping with the cognitive demands of attitude measures in surveys. Appl Cogn Psychol 1991, 5:213-236.
• [5]De Leeuw ED: Data Quality in Mail, Telephone, and Face to Face Surveys. Amsterdam: TT-Publikaties; 1992.
• [6]Kraus F, Steiner V: Modelling heaping effects in unemployment duration models - with an application to retrospective event data in the German socio-economic panel. Zentrum für Europäische Wirtschaftsforschung 1995. Discussion Paper No. 95–09
• [7]Wolff A: Heaping and its Consequences for Duration Analysis. Institut für Statistik, Ludwig-Maximilians-Universität München: Sonderforschungsbereich 386; 2000. Paper 203
• [8]De Leeuw E, Hox JJ: Internet surveys as part of a mixed-mode design. In Social and Behavioral Research and the Internet. Edited by Das M, Ester P, Kaczmirek L. New York, London: Routledge Taylor & Francis Group; 2011.
• [9]De Leeuw E: Choosing the method of data collection. In International Handbook of Survey Methodology. Edited by De Leeuw E, Hox JJ, Dillman D. New York: Taylor & Francis, Prychology Press; 2008:113-135. [EAM series]
• [10]Krosnick JA, Alwin DF: An evaluation of a cognitive theory of response-order effects in survey measurement. Public Opin Q 1987, 51:201-219.
• [11]Christian LM: How Mixed-Mode Surveys are Transforming Social Research: The Influence of Survey Mode on Measurement in web and Telephone Surveys. Washington State University; 2007.
• [12]Macer T: Weaving, not drowning: an update on take-up and best practices in mixed- and multi-mode research. 2005. [http://www.meaning.uk.com/resources/articles_papers/files/spss_directions_2005.pps webcite]
• [13]Revilla M: Quality in unimode and mixed-mode designs: a multitrait-multimethod approach. Surv Res Methods 2010, 4:151-164.
• [14]Link MW, Mokdad A: Advance letters as a means of improving respondent cooperation in random digit dial studies: a multistate experiment. Public Opin Q 2005, 69:572-587.
• [15]Bäckström C, Nilsson C: Mixed mode surveying. A comparison of paper-questionnaires and web-questionnaires. In Book Mixed Mode Surveying. A Comparison of Paper-Questionnaires and web-Questionnaires. City: Department of Information Technology and Media, Mid Sweden University; 2002.
• [16]De Bernardo DH, Curtis A: Using online and paper surveys: the effectiveness of mixed-mode methodology for populations over 50. Res Aging 2012.
• [17]Smith AB, King M, Butow P, Olver I: A comparison of data quality and practicality of online versus postal questionnaires in a sample of testicular cancer survivors. Psycho-Oncology 2013, 22:233-237.
• [18]McCabe SE, Diez A, Boyd CJ, Nelson TF, Weitzman ER: Comparing web and mail responses in a mixed mode survey in college alcohol use research. Addict Behav 2006, 31:1619-1627.
• [19]Christian LM, Dillman D, Smyth JD: The effects of mode and format on answers to scalar questions in telephone and web surveys. In Advances in telephone survey methodology. Edited by Lepkowski JM, Tucker C, Brick JM, De Leeuw E, Japec L, Lavrakas PJ. Hoboken, NJ: Wiley; 2008:250-275.
• [20]Lugtig PJ, Lensvelt-Mulders GJLM, Frerichs R, Greven F: Estimating nonresponse bias and mode effects in a mixed-mode survey. Int J Mark Res 2011, 53:669-686.
• [21]Ye C, Fulton J, Tourangeau R: More positive or more extreme? A meta-analysis of mode differences in response choice. Public Opin Q 2011, 75:349-365.
• [22]Ravens-Sieberer U, Erhart M, Wetzel R, Krügel A, Brambosch A: Phone respondents reported less mental health problems whereas mail interviewee gave higher physical health ratings. J Clin Epidemiol 2008, 61:1056-1060.
• [23]Feveile H, Olsen O, Hogh A: A randomized trial of mailed questionnaires versus telephone interviews: response patterns in a survey. BMC Med Res Methodol 2007, 7:27. BioMed Central Full Text
• [24]Kraus L, Pabst A: Studiendesign und methodik des epidemiologischen suchtsurveys 2009. [study design and methodology of the 2009 epidemiological survey of substance abuse]. Sucht 2010, 56:315-326.
• [25]Christensen AI, Ekholm O, Glümer C, Juel K: Effect of survey mode on response patterns: comparison of face-to-face and self-administered modes in health surveys. The: European Journal of Public Health; 2013.
• [26]Bowling A: Mode of questionnaire administration can have serious effects on data quality. J Public Health 2005, 27:281-291.
• [27]Klausch T, Hox JJ, Schouten B: Assessing the Mode-Dependency of Sample Selectivity Across the Survey Response Process. Statistics Netherlands: The Hague; 2013.
• [28]Jäckle A, Roberts C, Lynn P: Telephone Versus Face-to-Face Interviewing: Mode Effects on Data Quality and Likely Causes. Report on Phase II of the ESS-Gallup Mixed Mode Methodology Projekt. Colchester: University of Essex; 2006.
• [29]Béland Y, St-Pierre M: Mode effects in the Canadian community health survey: a comparison of CATI and CAPI. In Advances in Telephone Survey Methodology. Edited by Lepkowski JM, Tucker C, Brick JM, De Leeuw ED, Japec L, Lavrakas PJ, Link MW, Sangster RL. Hoboken (New Jersey): John Wiley & Sons; 2007:297-314.
• [30]Heerwegh D: Mode differences between face-to-face and web surveys: an experimental investigation of data quality and social desirability effects. Int J Public Opin Res 2009, 21:111-121.
• [31]Robert Koch-Institut: Daten und Fakten: Ergebnisse der Studie Gesundheit in Deutschland Aktuell 2009. [Data and Facts: Results of the German Health Update” Study 2009]. Beiträge zur Gesundheitsberichterstattung des Bundes. Berlin: Robert Koch-Institut; 2011.
• [32]Robert Koch-Institut: Daten und Fakten: Ergebnisse der Studie Gesundheit in Deutschland Aktuell 2010. [Data and Facts: Results of the “German Health Update” Study 2010]. Beiträge zur Gesundheitsberichterstattung des Bundes. Berlin: Robert Koch-Institut; 2012.
• [33]American Association for Public Opinion Research (AAPOR): Standard Definitions: Final Dispositions of Case Codes and Outcome Rates for Surveys (Revised 2011). Deerfield: AAPOR; 2011.
• [34]World Health Organization: Obesity - Preventing and Managing the Global Epidemic. Geneva: World Health Organization; 2000. WHO Technical Report Series, No. 894
• [35]Cox B, Van Oyen H, Cambois E, Jagger C, Le Roy S, Robine JM, Romieu I: The reliability of the minimum european health module. Int J Public Health 2009, 54:55-60.
• [36]Kroenke K, Strine TW, Spitzer RL, Williams JB, Berry JT, Mokdad AH: The PHQ-8 as a measure of current depression in the general population. J Affect Disord 2009, 114:163-173.
• [37]Washington Group on Disability Statistics (WG), Budapest Initiative (BI), United Nations Economic & Social Commission for Asia & the Pacific (UNESCAP): Development of disability measures for surveys: the extended set on functioning. http://www.cdc.gov/nchs/data/washington_group/Development_of_Disability_Measures_for_Surveys_The_Extended_Set_on_Functioning.pdf webcite
• [38]Bech P, Olsen LR, Kjoller M, Rasmussen NK: Measuring well-being rather than the absence of distress symptoms: a comparison of the SF-36 mental health subscale and the WHO-five well-being scale. Int J Methods Psychiatr Res 2003, 12:85-91.
• [39]Meltzer H: Development of a common instrument for mental health. In EUROHIS: Developing Common Instruments for Health Surveys. Edited by Nosikov A, Gudex C. Amsterdam: IOS Press; 2003.
• [40]Kilpeläinen K, Arpo A, ECHIM Core Group: European Health Indicators. Development and Initial Implementation. Helsinki: National Public Health Institute; 2008.
• [41]Bush K, Kivlahan DR, McDonell MB, Fihn SD, Bradley KA: The AUDIT alcohol consumption questions (AUDIT-C): an effective brief screening test for problem drinking. Arch Intern Med 1998, 158:1789-1795.
• [42]Reinert DF, Allen JP: The alcohol use disorders identification test: an update of research findings. Alcohol Clin Exp Res 2007, 31:185-199.
• [43]Mensink GBM, Lampert T, Bergmann E: Übergewicht und Adipositas in Deutschland 1984–2003 [Overweight and obesity in Germany 1984–2003]. Bundesgesundheitsbl 2005, 48:1348-1356.
• [44]Brauns H, Scherer S, Steinmann S: The CASMIN educational classification in international comparative research. In Advances in Cross-National Comparison. Edited by Hoffmeyer-Zlotnik JHP, Wolf C. New York: Kluwer; 2003:221-244.
• [45]Graubard BI, Korn EL: Predictive margins with survey data. Biometrics 1999, 55:652-659.
• [46]Shim JM, Shin E, Johnson TP: Self-rated health assessed by web versus mail modes in a mixed mode survey: the digital divide effect and the genuine survey mode effect. Med Care 2013, 51:774-781.
• [47]Fowler FJ, Roman AM, Xiao Di Z: Mode effects in a survey of medicare prostate surgery patients. Public Opin Q 1998, 62:29-46.
• [48]Hanmer J, Hays RD, Fryback DG: Mode of administration is important in US national estimates of health-related quality of life. Med Care 2007, 45:1171-1179.
• [49]McHorney CA, Kosinski M, Ware JE Jr: Comparisons of the costs and quality of norms for the SF-36 health survey collected by mail versus telephone interview: results from a national survey. Med Care 1994, 32:551-567.
• [50]Epstein JF, Barker PR, Kroutil LA: Mode effects in self-reported mental health data. Public Opin Q 2001, 65:529-549.
• [51]Gwaltney CJ, Shields AL, Shiffman S: Equivalence of electronic and paper-and-pencil administration of patient-reported outcome measures: a meta-analytic review. Value Health 2008, 11:322-333.
• [52]Tipping S, Hope S, Pickering K, Erens B, Roth MA, Mindell JS: The effect of mode and context on survey results: analysis of data from the health survey for England 2006 and the boost survey for London. BMC Med Res Methodol 2010, 10:84. BioMed Central Full Text
• [53]Link MW, Mokdad AH: Effects of survey mode on self-reports of adult alcohol consumption: a comparison of mail, web and telephone approaches. J Stud Alcohol 2005, 66:239-245.
• [54]Aquilino WS: Interview mode effects in surveys of drug and alcohol use: a field experiment. Public Opin Q 1994, 58:210-240.
• [55]Finger J, Tylleskar T, Lampert T, Mensink G: Physical activity patterns and socioeconomic position: the German National Health Interview and Examination Survey 1998 (GNHIES98). BMC Public Health 2012, 12:1079. BioMed Central Full Text
• [56]Biemer PP: Total survey error: design, implementation, and evaluation. Public Opin Q 2010, 74:817-848.
• [57]Groves RM, Lyberg L: Total survey error: past, present, and future. Public Opin Q 2010, 74:849-879.