Measurement properties of a novel survey to assess stages of organizational readiness for evidence-based interventions in community chronic disease prevention settings

Implementation Science

Douglas A Luke⁴  Ross C Brownson²  Mariah Dreisinger¹  Elizabeth Boland²  Carl Filler²  Amy McQueen⁵  Katherine A Stamatakis³ 

[1]School of Medicine, University of Missouri-Columbia, Columbia, MO, USA

[2]Prevention Research Center in St. Louis, Brown School, Washington University, St. Louis, MO, USA

[3]Division of Public Health Sciences and Alvin J. Siteman Cancer Center, Washington University School of Medicine, St. Louis, MO, USA

[4]Center for Tobacco Policy Research, Brown School, Washington University, St. Louis, MO, USA

[5]Division of Health Behavior Research, Washington University School of Medicine, St. Louis, MO, USA

关键词: Implementation;    Dissemination;    Confirmatory factor analysis;    Evidence-based practice;    Chronic disease prevention;    Measurement tool;   

Others  :  814784
DOI  :  10.1186/1748-5908-7-65

 received in 2011-06-01, accepted in 2012-07-16,  发布年份 2012

【 摘 要 】

Background

There is a great deal of variation in the existing capacity of primary prevention programs and policies addressing chronic disease to deliver evidence-based interventions (EBIs). In order to develop and evaluate implementation strategies that are tailored to the appropriate level of capacity, there is a need for an easy-to-administer tool to stage organizational readiness for EBIs.

Methods

Based on theoretical frameworks, including Rogers’ Diffusion of Innovations, we developed a survey instrument to measure four domains representing stages of readiness for EBI: awareness, adoption, implementation, and maintenance. A separate scale representing organizational climate as a potential mediator of readiness for EBIs was also included in the survey. Twenty-three questions comprised the four domains, with four to nine items each, using a seven-point response scale. Representatives from obesity, asthma, diabetes, and tobacco prevention programs serving diverse populations in the United States were surveyed (N = 243); test-retest reliability was assessed with 92 respondents.

Results

Confirmatory factor analysis (CFA) was used to test and refine readiness scales. Test-retest reliability of the readiness scales, as measured by intraclass correlation, ranged from 0.47–0.71. CFA found good fit for the five-item adoption and implementation scales and resulted in revisions of the awareness and maintenance scales. The awareness scale was split into two two-item scales, representing community and agency awareness. The maintenance scale was split into five- and four-item scales, representing infrastructural maintenance and evaluation maintenance, respectively. Internal reliability of scales (Cronbach’s α) ranged from 0.66–0.78. The model for the final revised scales approached good fit, with most factor loadings >0.6 and all >0.4.

Conclusions

The lack of adequate measurement tools hinders progress in dissemination and implementation research. These preliminary results help fill this gap by describing the reliability and measurement properties of a theory-based tool; the short, user-friendly instrument may be useful to researchers and practitioners seeking to assess organizational readiness for EBIs across a variety of chronic disease prevention programs and settings.

【 授权许可】

   
2012 Stamatakis et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Xu J, Kochanek KD, Murphy SL, Tejada-Vera B: Deaths: final data for 2007. National vital statistics reports, vol. 58 (19), Hyattsville, MD: National Center for Health Statistics; 2010.
• [2]Anderson G, Herbert R, Zeffiro T, Johnson N: Chronic Conditions: Making the Case for Ongoing Care. Partnership for Solutions, Johns Hopkins University, Baltimore; 2004.
• [3]Flegal KM, Graubard BI, Williamson DF, Gail MH: Excess deaths associated with underweight, overweight, and obesity. JAMA 2005, 293:1861-1867.
• [4]Mokdad AH, Marks JS, Stroup DF, Gerberding JL: Actual causes of death in the United States, 2000. JAMA 2004, 291:1238-1245.
• [5]Murray CJ, Kulkarni SC, Ezzati M: Understanding the coronary heart disease versus total cardiovascular mortality paradox: a method to enhance the comparability of cardiovascular death statistics in the United States. Circulation 2006, 113:2071-2081.
• [6]Cancer Control PLANEThttp://cancercontrolplanet.cancer.gov/index.html webcite
• [7]Research-tested Interventions Programshttp://rtips.cancer.gov/rtips/index.do webcite
• [8]Guide to Community Preventive Serviceshttp://www.thecommunityguide.org/index.html webcite
• [9]Guide to Clinical Preventive Services 2010-2011. http://www.ahrq.gov/clinic/pocketgd.htm webcite
• [10]Brownson RC, Ballew P, Brown KL, Elliott MB, Haire-Joshu D, Heath GW, Kreuter MW: The effect of disseminating evidence-based interventions that promote physical activity to health departments. Am J Public Health 2007, 97:1900-1907.
• [11]Zaza S, Briss PA, Harris KW (Eds): The Guide to Community Preventive Services: What Works to Promote Health?. Oxford University Press, New York; 2005.
• [12]Brownson RC, Fielding JE, Maylahn CM: Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health 2009, 30:175-201.
• [13]Dearing JW: Evolution of diffusion and dissemination theory. J Public Health Manag Pract 2008, 14:99-108.
• [14]Dearing JW, Kee KF: Historical Roots of Dissemination and Implementation Science. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson R, Colditz G, Proctor E. Oxford University Press, Oxford; 2012. in press
• [15]Gaglio B, Glasgow RE: Evaluation Approaches for Dissemination and Implementation Research. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson R, Colditz G, Proctor E. Oxford University Press, Oxford; 2012. in press
• [16]Rogers EM: Diffusion of innovations. Free Press, New York; 2003.
• [17]Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999, 89:1322-1327.
• [18]Balbach ED: Using case studies to do program evaluation. California Department of Health Services, Sacramento; 1999.
• [19]Briss PA, Brownson RC, Fielding JE, Zaza S: Developing and using the Guide to Community Preventive Services: lessons learned about evidence-based public health. Annu Rev Public Health 2004, 25:281-302.
• [20]Dreisinger ML, Boland EM, Filler CD, Baker EA, Hessel AS, Brownson RC: Contextual factors influencing readiness for dissemination of obesity prevention programs and policies. Heal Educ Res 2012, 27:292-306.
• [21]Steckler A, Goodman RM, McLeroy KR, Davis S, Koch G: Measuring the diffusion of innovative health promotion programs. Am J Health Promot 1992, 6:214-224.
• [22]Jacobs JA, Dodson EA, Baker EA, Deshpande AD, Brownson RC: Barriers to evidence-based decision making in public health: a national survey of chronic disease practitioners. Public Health Rep 2010, 125:736-742.
• [23]Center for Tobacco Policy Research: Program Sustainability Assessment Tool. Center for Tobacco Policy Research, Brown School, Washington University in St. Louis, St. Louis; 2011. http://ctpr.wustl.edu/documents/Sustainability_Tool_3.11.pdf webcite
• [24]Jobe JB, Mingay DJ: Cognitive research improves questionnaires. Am J Public Health 1989, 79:1053-1055.
• [25]Jobe JB, Mingay DJ: Cognitive laboratory approach to designing questionnaires for surveys of the elderly. Public Health Rep 1990, 105:518-524.
• [26]Streiner D, Norman G: Health Measurement Scales: A practical guide to their development and use. 3rd edition. Oxford University Press, New York; 2006.
• [27]Willis GB: Cognitive interviewing: a tool for improving questionnaire design. Sage, Thousand Oaks; 2005.
• [28]Luke DA, Ribisl KM, Walton MA, Davidson WS: Assessingthe diversity of personal beliefs about addiction: Development of the Addiction Belief Inventory. Substance Use Misuse 2002, 37:91-121.
• [29]Maruyama GM: Basics of structural equation modeling. Sage Publications, Thousand Oaks, CA; 1998.
• [30]Arbuckle JL: Amos 16.0 User's Guide. Amos Development Corporation, Chicago; 2007.
• [31]Arbuckle JL, Marcoulides GA, Schumacker RE: Full information estimation in the presence of incomplete data. In Advanced structural equation modeling: issues and techniques. Erlbaum, Mahwah; 1996:243-277.
• [32]Schafer JL, Graham JW: Missing data: Our view of the state of the art. Psychol Meth 2002, 7:147-177.
• [33]Hu L, Bentler PM: Evaluating model fit. In Structural equation modeling. Edited by Hoyle RH. Sage Publications, Thousand Oaks; 1995:76-99.
• [34]Hu L, Bentler PM: Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Model 1999, 6:1-55.
• [35]StataCorp: Stata Statistical Software: Release 11. In Book Stata Statistical Software: Release 11. StataCorp LP, College Station, TX; 2009.
• [36]Rousson V, Gasser T, Seifert B: Assessing intrarater, interrater and test-retest reliability of continuous measurements. Stat Med 2002, 21:3431-3446.
• [37]Rosner B: Fundamentals of Biostatistics. 7th edition. Brooks/Cole, Boston; 2010.
• [38]Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004, 82:581-629.
• [39]Dobbins M, Hanna SE, Ciliska D, Manske S, Cameron R, Mercer SL, O'Mara L, DeCorby K, Robeson P: A randomized controlled trial evaluating the impact of knowledge translation and exchange strategies. Implement Sci 2009, 4:61. BioMed Central Full Text
• [40]Brownson RC, Ballew P, Dieffenderfer B, Haire-Joshu D, Heath GW, Kreuter MW, Myers BA: Evidence-based interventions to promote physical activity: what contributes to dissemination by state health departments. Am J Prev Med 2007, 33:S66-S73. quiz S74-68
• [41]Aarons GA, Horowitz JD, Dlugosz LR, Ehrhart MG: The role of organizational processes in dissemination and implementation research. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson RC, Colditz G, Proctor E. Oxford University Press, Oxford; in press
• [42]Glisson C, James LR: The cross-level effects of organizational climate and culture in human service teams. J Organ Behav 2002, 23:767-794.
• [43]Weiner BJ, Amick H, Lee SY: Conceptualization and measurement of organizational readiness for change: a review of the literature in health services research and other fields. Med Care Res Rev 2008, 65:379-436.
• [44]Weiner BJ: A theory of organizational readiness for change. Implement Sci 2009, 4:67. BioMed Central Full Text
• [45]Stamatakis KA, Vinson CA, Kerner JF: Dissemination and implementation research in community and public health settings. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson RC, Colditz GA, Proctor EK. Oxford University Press, New York; 2012.
• [46]Green LW: Public health asks of systems science: to advance our evidence-based practice, can you help us get more practice-based evidence? Am J Public Health 2006, 96:406-409.
• [47]Proctor E, Brownson RC: Measurement Issues in Dissemination and Implementation Research. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson RC, Colditz G, Proctor E. Oxford University Press; in press
• [48]Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence-based quality improvement. Evidence (and its limitations) of the effectiveness of guideline dissemination and implementation strategies 1966-1998. J Gen Intern Med 2006, 21(Suppl 2):S14-S20.
• [49]Cargo M, Mercer SL: The value and challenges of participatory research: strengthening its practice. Annu Rev Public Health 2008, 29:325-350.
• [50]Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A, Griffey R, Hensley M: Outcomes for Implementation Research: Conceptual Distinctions, Measurement Challenges, and Research Agenda. Adm Policy Ment Health 2010, 38:65-76.