Comparison of response rates and cost-effectiveness for a community-based survey: postal, internet and telephone modes with generic or personalised recruitment approaches

BMC Medical Research Methodology

Karin Leder¹  Manori Malawaraarachchi¹  Joanne O’Toole²  Martha Sinclair¹ 

[1] Department of Epidemiology and Preventive Medicine, School of Public Health and Preventive Medicine, Monash University, Melbourne, Australia;School of Land and Environment, The University of Melbourne, Parkville, Australia

关键词: Cost-effectiveness;    Internet survey;    Telephone survey;    Postal survey;    Survey methods;   

Others  :  1126880
DOI  :  10.1186/1471-2288-12-132

 received in 2012-01-25, accepted in 2012-08-27,  发布年份 2012

【 摘 要 】

Background

Epidemiological research often requires collection of data from a representative sample of the community or recruitment of specific groups through broad community approaches. The population coverage of traditional survey methods such as mail-outs to residential addresses, and telephone contact via public directories or random-digit-dialing is declining and survey response rates are falling. There is a need to explore new sampling frames and consider multiple response modes including those offered by changes in telecommunications and internet technology.

Methods

We evaluated response rates and cost-effectiveness for three modes of survey administration (postal invitation/postal survey, postal invitation/internet survey and postal invitation/telephone survey) and two styles of contact approach (personalised and generic) in a community survey of greywater use. Potential respondents were contacted only once, with no follow up of non-responders.

Results

The telephone survey produced the highest adjusted response rate (30.2%), followed by the personalised postal survey (10.5%), generic postal survey (7.5%) and then the internet survey (4.7% for the personalised approach and 2.2% for the generic approach). There were some differences in household characteristics and greywater use rates between respondents to different survey modes, and between respondents to personalised and generic approaches. These may be attributable to the differing levels of motivations needed for a response, and varying levels of interest in the survey topic among greywater users and non-users. The generic postal survey had the lowest costs per valid survey received (Australian $22.93), followed by the personalised postal survey ($24.75).

Conclusions

Our findings suggest that postal surveys currently remain the most economic option for population-based studies, with similar costs for personalised and generic approaches. Internet surveys may be effective for specialised groups where email lists are available for initial contact, but barriers other than household internet access still exist for community-based surveys. Given the increasing recruitment challenges facing community-based studies, there is an imperative to gather contemporary comparative data on different survey modes and recruitment approaches in order to determine their strengths, limitations and costs. Researchers also need to document and report on the potential biases in the target and respondent populations and how this may affect the data collected.

【 授权许可】

   
2012 Sinclair et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Blumberg SJ, Luke JV: Wireless substitution: early release of estimates from the National Health Interview Survey, July–December 2009. Centers for Disease Control and Prevention; 2010. http://www.cdc.gov/nchs/nhis/releases.htm#wireless webcite
• [2]Blumberg SJ, Luke JV: Wireless substitution: Early release of estimates from the National Health Interview Survey, January-June 2011. Centers for Disease Control and Prevention; 2011. http://www.cdc.gov/nchs/nhis/releases.htm#wireless webcite
• [3]Guterbock TM, Diop A, Ellis JM, Holmes JL, Le KT: Who needs RDD? Combining directory listings with cell phone exchanges for an alternative telephone sampling frame. Soc Sci Res 2011, 40(3):860-872.
• [4]Galea S, Tracy M: Participation rates in epidemiologic studies. Ann Epidemiol 2007, 17:643-653.
• [5]Curtin R, Presser S, Singer E: Changes in Telephone Survey Nonresponse over the Past Quarter Century. Public Opin Q 2005, 69(1):87-98.
• [6]Ekman A, Litton J-E: New times, new needs; e-epidemiology. Eur J Epidemiol 2007, 22:285-292.
• [7]Van Gelder MMHJ, Bretveld RW, Roeleveld N: Web-based questionnaires: the future in epidemiology? Am J Epidemiol 2010, 172(11):1292-1298.
• [8]Link MW, Mokdad AH: Alternative modes for health surveillance surveys: an experiment with web, mail and telephone. Epidemiology 2005, 16(5):701-704.
• [9]Källmén H, Sinadinovic K, Berman AH, Wennberg P: Risky drinking of alcohol in Sweden: a randomized population survey comparing web- and paper-based self-reports. NAT Nordic Stud Alcohol and Drugs 2011, 28(2):123-130.
• [10]Eriksen L, Grønbæk M, Helge JW, Tolstrup JS, Curtis T: The Danish health examination survey 2007–2008 (DANHES 2007–2008). Scand J Public Health 2011, 39(2):203-211.
• [11]Manfreda KL, Bosnjak M, Berzelak J, Haas I, Vehovar V: Web surveys versus other survey modes. A meta-analysis comparing response rates. Int J Market Res 2008, 50(1):79-104.
• [12]Shih TH, Xitao F: Comparing response rates from web and mail surveys: A meta-analysis. Field Methods 2008, 20(3):249-271.
• [13]Balter KA, Balter O, Fondell E, Lagerros YT: Web-based and mailed questionnaires: A comparison of response rates and compliance. Epidemiology 2005, 16(4):577-579.
• [14]Ekman A, Dickman PW, sa Klint A, Weiderpass E, Litton JE: Feasibility of using web-based questionnaires in large population-based epidemiological studies. Eur J Epidemiol 2006, 21:103-111.
• [15]Couper MP: The future of modes of data collection. Public Opin Q 2011, 75(5 SPEC):889-908.
• [16]Blyth B: Mixed mode: The only ‘fitness’ regime? Int J Market Res 2008, 50(2):241-266.
• [17]Brogger J, Nystad W, Cappelen I, Bakke P: No increase in response rate by adding a web response option to a postal population survey: a randomized trial. J Med Internet Res 2007, 9(5):e40.
• [18]Turunen M, Paanala A, Villman J, Nevalainen A, Haverinen-Shaughnessy U: Evaluating housing quality, health and safety using an Internet-based data collection and response system: a cross-sectional study. Environ Health 2010, 9:69. BioMed Central Full Text
• [19]Messer BL, Dillman DA: Surveying the general public over the internet using address-based sampling and mail contact procedures. Public Opin Q 2011, 75(3):429-457.
• [20]Dillman DA, Phelps G, Tortora R, Swift K, Kohrell J, Berck J, Messer BL: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc Sci Res 2009, 38(1):1-18.
• [21]Fan W, Yan Z: Factors affecting response rates of the web survey: a systematic review. Comput Hum Behav 2010, 26:132-139.
• [22]Edwards PJ, Roberts I, Clarke MJ, DiGuiseppi C, Wentz R, Kwan I, Cooper R, Felix LM, Pratap S: Methods to increase response to postal and electronic questionnaires. Cochrane Database Syst Rev 2009, Issue 3:Art. No.:MR000008.
• [23]Dillman DA, Lesser V, Mason R, Carlson J, Willits F, Robertson R, Burke B: Personalization of mail surveys for general public and populations with a group identity: results from nine studies. Rural Sociol 2007, 72(4):632-646.
• [24]Heerwegh D: Effects of personal salutations in e-mail invitations to participate in a web survey. Public Opin Q 2005, 69(4):588-598.
• [25]Muñoz-Leiva F, Sánchez-Fernández J, Montoro-Ríos F, Ibáñez-Zapata JA: Improving the response rate and quality in Web-based surveys through the personalization and frequency of reminder mailings. Qual Quant 2010, 44(5):1037-1052.
• [26]Sauermann H, Roach M: Increasing web survey response rates in innovation research: An experimental study of static and dynamic contact design features. Res Policy 2012. http://dx.doi.org/10.1016/j.respol.2012.05.003 webcite
• [27]Gajic A, Cameron D, Hurley J: The cost-effectiveness of cash versus lottery incentives for a web-based, stated-preference community survey. Eur J Health Econ 2011, 1-11.
• [28]Göritz AS: Incentives in web studies: methodological issues and a review. Int J Internet Sci 2006, 1(1):58-70.
• [29]ABS: Socio-Economic Indexes for Areas. Australian Bureau of Statistics; 2008. http://www.abs.gov.au/websitedbs/D3310114.nsf/home/Seifa_entry_page webcite
• [30]ABS: Census Home Page. Australian Bureau of Statistics; 2006. http://www.abs.gov.au/websitedbs/censushome.nsf/home/Census webcite
• [31]De Leeuw E, Callegaro M, Hox J, Korendijk E, Lensvelt-Mulders G: The influence of advance letters on response in telephone surveys. A meta-analysis. Public Opin Q 2007, 71(3):413-443.
• [32]McCluskey S, Topping AE: Increasing response rates to lifestyle surveys: a pragmatic evidence review. Perspect Public Health 2011, 131(2):89-94.
• [33]ABS: Environmental Issues: Water use and conservation. Australian Bureau of Statistics; 2010. http://www.abs.gov.au/ausstats/abs@.nsf/mf/4602.0.55.003/ webcite
• [34]Mokdad AH: The behavioral risk factors surveillance system: Past, present, and future. Annu Rev Public Health 2009, 30:43-54.
• [35]Fenner Y, Garland SM, Moore EE, Jayasinghe Y, Fletcher A, Tabrizi SN, Gunasekaran B, Wark JD: Web-based recruiting for health research using a social networking site: an exploratory study. J Med Internet Res 2012, 14(1):e20.
• [36]Ramo DE, Hall SM, Prochaska JJ: Reaching young adult smokers through the Internet: Comparison of three recruitment mechanisms. Nicotine Tob Res 2010, 12(7):768-775.
• [37]Groves RM: Nonresponse rates and nonresponse bias in household surveys. Public Opin Q 2006, 70(5):646-675.