An intervention to improve program implementation: findings from a two-year cluster randomized trial of Assets-Getting To Outcomes

Implementation Science

Mary Ellen Slaughter²  Peter Scales¹  Andrea Phillips²  Susan Paddock²  Patrick S Malone³  Patricia Ebener²  Matthew Chinman²  Joie Acosta² 

[1] Search Institute, 615 First Avenue NE, Suite 125, Minneapolis MN 55413, USA;RAND Corporation, 4570 Fifth Avenue, Pittsburgh PA 15213, USA;University of South Carolina, 920 Sumter St, Columbia SC 29208, USA

关键词: Capacity building;    Technical assistance;    Implementation support;   

Others  :  813450
DOI  :  10.1186/1748-5908-8-87

 received in 2013-04-12, accepted in 2013-07-31,  发布年份 2013

【 摘 要 】

Background

Studies have shown that communities have not always been able to implement evidence-based prevention programs with quality and achieve outcomes demonstrated by prevention science. Implementation support interventions are needed to bridge this gap between science and practice. The purpose of this article is to present two-year outcomes from an evaluation of the Assets Getting To Outcomes (AGTO) intervention in 12 Maine communities engaged in promoting Developmental Assets, a positive youth development approach to prevention. AGTO is an implementation support intervention that consists of: a manual of text and tools; face-to-face training, and onsite technical assistance, focused on activities shown to be associated with obtaining positive results across any prevention program.

Methods

This study uses a nested and cross-sectional, cluster randomized controlled design. Participants were coalition members and program staff from 12 communities in Maine. Each coalition nominated up to five prevention programs to participate. At random, six coalitions and their respective 30 programs received the two-year AGTO intervention and the other six maintained routine operations. The study assessed prevention practitioner capacity (efficacy and behaviors), practitioner exposure to and use of AGTO, practitioner perceptions of AGTO, and prevention program performance. Capacity of coalition members and performance of their programs were compared between the two groups across the baseline, one-, and two-year time points.

Results

We found no significant differences between AGTO and control group’s prevention capacity. However, within the AGTO group, significant differences were found between those with greater exposure to and use of AGTO. Programs that received the highest number of technical assistance hours showed the most program improvement.

Conclusions

This study is the first of its kind to show that use of an implementation support intervention-AGTO -yielded improvements in practitioner capacity and consequently in program performance on a large sample of practitioners and programs using a randomized controlled design.

ClinicalTrials.gov identifier

NCT00780338

【 授权许可】

   
2013 Acosta et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

【 参考文献 】

• [1]Miller T, Hendrie D: Substance abuse prevention dollars and cents: A cost-benefit analysis (DHHS Pub. No. (SMA) 07–4298). Rockville, MD: Center for Substance Abuse Prevention, Substance Abuse and Mental Health Services Administration; 2009.
• [2]Backer TE: Finding the balance: program fidelity and adaptation in substance abuse prevention: CSAP. 2001.
• [3]Ennett ST, Ringwalt CL, Thorne J, Rohrbach LA, Vincus A, Simons-Rudolph A, et al.: A comparison of current practice in school-based substance abuse prevention programs with meta-analysis findings. Prev Sci 2003, 4:1-14.
• [4]Cantor D, Crosse S, Hagen CA, Mason MJ, Siler AJ, Von Glatz A: Wide Scope, Questionable Quality: Drug and Violence Prevention Efforts in American Schools. Report on the Study on School Violence and Prevention. In. Washington, DC: U.S. Department of Education, Planning and Evaluation Service; 2001.
• [5]Crosse S, Burr M, Cantor D, Hagen CA, Hantman I: Wide Scope, Questionable Quality: Drug and Violence Prevention Efforts in American Schools. Report on the Study on School Violence and Prevention. In. Washington, DC: U.S. Department of Education, Planning and Evaluation Service; 2001.
• [6]Reuter P, Timpane JM: Assessing options for the safe and drug free schools and communities act. Santa Monica, CA: RAND; 2001. No. MR-1328
• [7]Silvia ES, Thorne J: School-based drug prevention programs: A longitudinal study in selected school districts. In. North Carolina: Research Triangle Institute; 1997.
• [8]U.S. Department of Education OoP, Evaluation and Policy Development, Policy and Program Studies Service: Prevalence and Implementation Fidelity of Research-Based Prevention Programs in Public Schools: Final Report. Washington, D.C: ; 2011.
• [9]Wandersman A, Florin P: Community interventions and effective prevention: bringing researchers/evaluators, funders and practitioners together for accountability. Am Psychol 2003, 58(6/7):441-448.
• [10]Green L: From research to 'Best Practices' in other settings and populations. Am J Health Behav 2001, 25(3):165-178.
• [11]Livet M, Wandersman A: Organizational functioning: facilitating effective interventions and increasing the odds of programming success. In Empowerment evaluation in practice. Edited by Fetterman DM, Wandersman A. New York, NY: Guilford; 2005:123-154.
• [12]Chinman M, Hunter S, Ebener P, Paddock S, Stillman L, Imm P, Wandersman A: The Getting To Outcomes demonstration and evaluation: an illustration of the prevention support system. Am J Community Psychol 2008, 41:206-224.
• [13]Chinman M, Tremain B, Imm P, Wandersman A: Strengthening prevention performance using technology: a formative evaluation of interactive Getting To OutcomesTM. Am J Orthopsychiatry 2009, 79:469-481.
• [14]Fishbein M, Ajzen A: Beliefs, attitudes, intentions, and behavior: An introduction to theory and research. Reading, MA: Addison-Wesley; 1975.
• [15]Ajzen I, Fishbein M: Attitude-behavior relations: a theoretical analysis and review of empirical research. Psychol Bull 1977, 84:888-918.
• [16]Bandura A: Health promotion by social cognitive means. Health Educ Behav 2004, 31(2):143-164.
• [17]Fishbein M, Ajzen I: Atitudes toward objects as predictive of single and multiple behavioral criteria. Psychol Rev 1974, 81:59-74.
• [18]Chinman M, Acosta J, Ebener P, Burkhart Q, Clifford M, Corsello M, Duffy T, Hunter S, Jones M, Lahti M, et al.: Establishing and evaluating the key functions of an interactive systems framework based on Assets-Getting To Outcomes. Am J Commun Psychol 2012. http://rd.springer.com/article/10.1007%2Fs10464-012-9504-z webcite
• [19]Benson PL, Scales PC, Leffert N, Roehlkepartain ER: A fragile foundation: The state of developmental assets among American youth. Minneapolis, MN: Search Institute; 1999.
• [20]Leffert N, Benson PL, Scales PC, Sharma AR, Drake DR, Blyth DA: Developmental assets: measurement and prediction of risk behaviors among adolescents. Appl Dev Sci 1998, 2:209-230.
• [21]Scales PC, Benson PL, Leffert N, Blyth DA: Contribution of developmental assets to the prediction of thriving among adolescents. Appl Dev Sci 2000, 4:27-46.
• [22]Fisher D, Imm PS, Chinman M, Wandersman A: Getting To Outcomes with Developmental Assets: Ten steps to measuring success in youth programs and communities. Minneapolis, MN: Search Institute; 2006.
• [23]Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009, 4:50-65. BioMed Central Full Text
• [24]Rohrbach LA, D’Onofrio CN, Backer TE, Montgomery SB: Diffusion of school-based substance abuse prevention programs. Am Behav Sci 1996, 39:919-934.
• [25]Rohrbach LA, Graham JW, Hansen WB: Diffusion of a school-based substance abuse prevention program: predictors of program implementation. Prev Med 1993, 22(2):237-260.
• [26]McCormick LK, Steckler AB, McLeroy KR: Diffusion of innovations in schools: a study of adoption and implementation of school-based tobacco prevention curricula. Am J Health Promot 1995, 9:210-219.
• [27]Steckler AB, McLeroy KR, Goodman RM, McCormick L, Bird ST: Toward integrating qualitative and quantitative methods: an introduction. Health Educ Q 1992, 19:1-8.
• [28]Parcel GS, Erikson MP, Lovato CY, Gottlieb NH, Brink SG, Green LW: The diffusion of school-based tobacco-use prevention program: project description and baseline data. Health Educ Res 1989, 4:111-124.
• [29]Cividin TM, Ottoson JM: Linking reasons for continuing professional education participation with post-program application. J Contin Educ Health Professions 1997, 17:46-55.
• [30]Ottoson JM: After the applause: exploring multiple influences on application following an adult education program. Adult Educ Quart 1997, 47:92-107.
• [31]Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009, 4:1. BioMed Central Full Text
• [32]Murray DM: Design and analysis of group-randomized trials. Oxford, UK: Press OU; 1998.
• [33]Battelle: Annual findings report: Drug-free communities support program national evaluation. Washington, DC: Executive Office of the President Office of Administration Office of National Drug Control Policy; 2008.
• [34]Butterfoss FD, Goodman RM, Wandersman A: Community coalitions for prevention and health promotion. Health Educ Res 1993, 8:315-330.
• [35]Ellickson PL, Bell RM: Drug prevention in junior high: a multi-site longitudinal test. Science 1990, 247:1265-1372.
• [36]Botvin GJ, Baker E, Dusenbury L, Botvin EM, Diaz T: Long-term follow-up results of a randomized drug abuse prevention trial in a white middle-class population. JAMA 1995, 273(14):1106-1112.
• [37]Jemmott JB, Jemmott L, Spears H, Hewitt N, Cruz-Collins M: Self-efficacy, hedonistic expectancies, and condom-use intentions among inner-city Black adolescent women: a social cognitive approach to AIDS risk behavior. J Adolescent Health 1992, 13:512-519.
• [38]Jemmott JB, Jemmott LS, Fong GT: Reductions in HIV risk-associated sexual behaviors among Black male adolescents: effects of an AIDS prevention intervention. Am J Public Health 1992, 82:372-377.
• [39]Jemmott JB, Jemmott LS, Fong GT, McCaffree K: Reducing HIV risk-associated sexual behavior among African American adolescents: Testing the generality of intervention effects. Am J Community Psychol 1999, 27:161-187.
• [40]Chinman M, Hannah G, Wandersman A, Ebener P, Hunter S, Imm P, Sheldon J: Developing a community science research agenda for building community capacity for effective preventive interventions. Am J Community Psychol 2005, 3/4:143-157.
• [41]Mitchell R, Stone-Wiggins B, Stevenson JF, Florin P: Cultivating capacity: outcomes of a statewide support system for prevention coalitions. J Prev Interv Community 2004, 27:67-87.
• [42]O’Donnell LO, Scattergood P, Adler M, San Doval A, Barker M, Kelly JA, et al.: The role of technical assistance in the replication of effective HIV interventions. AIDS Educ Prev 2000, 12:99-111.
• [43]Stevenson JF, Florin P, Mills DS, Andrade M: Building evaluation capacity in human service organizations: a case study. Eval Program Plann 2002, 25(3):233-243.
• [44]Kitson A, Harvey G, McCormack B: Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998, 7:149-158.
• [45]Rycroft-Malone J, Harvey G, Seers K, Kitson A, McCormack B, Titchen A: An exploration of the factors that influence the implementation of evidence into practice. J Clin Nurs 2002, 13:913-924.
• [46]Stetler CB, Legro MW, Rycroft-Malone J, Bowman C, Curran G, Guihan M, et al.: Role of 'external facilitation' in implementation of research findings: a qualitative evaluation of facilitation experiences in the Veterans Health Administration. Implement Sci 2006., 23
• [47]Weiss E, Miller-Anderson R, Lasker R: Making the most of collaboration: exploring the relationship between partnership synergy and partnership functioning. Health Educ Behav 2002, 29:683-698.
• [48]Lehman WEK, Greener JM, Simpson DD: Assessing organizational readiness for change. J Subst Abuse Treat 2002, 22(4):197-209.
• [49]Hall GE, Hord SM: Implementing change. Boston, MA: Allyn and Bacon; 2001.
• [50]Hall GE, Loucks SF, Rutherford WL, Newlove BW: Levels of use of the innovation: a framework for analyzing innovation adoption. J Teach Educ 1975, 26:52.
• [51]Hall GE, Loucks SF: A developmental model for determining whether the treatment is actually implemented. Am Educ Res J 1977, 14(3):263-276.
• [52]Chinman M, Acosta J, Ebener P, Burkhart Q, Clifford M, Corsello M, Duffy T, Hunter S, Jones M, Lahti M, et al.: Intervening with practitioners to improve the quality of prevention: One year findings from a randomized trial of Assets-Getting To Outcomes. J Prim PrevIn press
• [53]Chinman M, Tremain B, Imm P, Wandersman A: Strengthening prevention performance using technology: a formative evaluation of interactive Getting To Outcomes. Am J Orthopsychiatry 2009, 79:469-481. PMC2859836
• [54]Bryt T, Bishop J, Carlin JB: Bias, prevalence and kappa. J Clin Epidemiol 1993, 46(5):423-429.
• [55]Lincoln YS, Guba EG: Naturalistic inquiry. In. Beverly Hills, CA: Sage; 1985.
• [56]Graham ID, Logan J: Innovations in knowledge transfer and continuity of care. Can J Nurs Res 2004, 36(2):89-103.
• [57]Hunter SB, Chinman M, Ebener P, Imm P, Wandersman A, Ryan G: Technical assistance as a prevention capacity-building tool: a demonstration using the Getting To Outcomes framework. Health Educ Behav 2009, 36:810-828.
• [58]Keener DC: Toward a science of capacity building: An examination of technical assistance following a training program for prevention professionals. In. Columbia, SC, USA: University of South Carolina; 2007.
• [59]Feinberg ME, Ridenour TA, Greenberg MT: The longitudinal effect of technical assistance dosage on the functioning of communities that care prevention boards in Pennsylvania. J Prim Prev 2008, 29(2):145-165.
• [60]Hawkins JD, Oesterle S, Brown EC, Arthur MW, Abbott RD, Fagan AA, Catalano RF: Results of a type 2 translational research trial to prevent adolescent drug use and delinquency: a test of communities that care. Arch Pediatr Adolesc Med 2009 September, 163(9):789-798.
• [61]Spoth R, Redmond C, Shin C, Greenberg M, Clair S, Feinberg M: Substance-use outcomes at 18 months past baseline: the PROSPER Community-University partnership trial. Am J Prev Med 2007 May, 32(5):395-402.
• [62]Wandersman A: Four keys to success (theory, implementation, evaluation, and resource/system support): high hopes and challenges in participation. Am J Community Psychol 2009, 43:3-21.
• [63]Wandersman A, Chien VH, Katz J: Toward an evidence-based system for innovation support for implementing innovations with quality: tools, training, technical assistance, and quality assurance/quality improvement. In Cultivating communities of practice. Edited by Wenger E, McDermott R, Snyder WM. Boston, MA: Harvard Business School Press; 2002. In press