Measurement of implementation components ten years after a nationwide introduction of empirically supported programs – a pilot study

Implementation Science

Nina Tollefsen¹  Knut Taraldsen¹  Terje Christiansen¹  Joshua Patras¹  John Kjøbli¹  Gunnar Bjørnebekk¹  Terje Ogden¹ 

[1] Norwegian Center for Child Behavioral Development, University of Oslo, P.O. Box 7053, Majorstuen, 0306, Oslo, Norway

关键词: Multisystemic therapy;    Parent management training;    Empirically supported treatment programs;    Implementation components;    Implementation measurement;   

Others  :  815387
DOI  :  10.1186/1748-5908-7-49

 received in 2011-10-11, accepted in 2012-05-31,  发布年份 2012

【 摘 要 】

Background

Ten years after the nationwide dissemination of two evidence-based treatment programs, the status of the implementation components was evaluated in a cross-sectional study. The aim of the study was to pilot a standardized measure of implementation components by examining the factor structure, the reliabilities of the scores, and their association with implementation outcome variables. The aim was also to compare implementation profiles of the two evidence-based programs based on multi informant assessments.

Methods

The 218 participants in the study were therapists, supervisors, and agency leaders working with Parent Management Training, the Oregon model (PMTO), and Multisystemic Therapy (MST) in Norway. Interviewers filled in an electronic version of the Implementation Components Questionnaire during a telephone interview.

Results

The factor analysis of the eight one-dimensional subscales resulted in an individual clinical-level factor and an organizational system-level factor. Age, experience, and number of colleagues in the workplace were negatively correlated with positive ratings of the implementation process, but the number of colleagues working with the same program predicted positive ratings. MST and PMTO had different implementation profiles and therapists, supervisors, and managers evaluated some of the implementation drivers significantly differently.

Conclusions

The psychometric quality of the questionnaire was supported by measures of internal consistency, factor analyses of the implementation components, and the comparisons of implementation profiles between programs and respondent groups. A moderate, but consistent association in the expected direction was found with the implementation outcome variables.

【 授权许可】

   
2012 Ogden et al; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Q 2004, 82:581-629.
• [2]Klein KJ, Sorra JS: The challenge of innovation implementation. Acad Manage Rev 1996, 21:1055-1080.
• [3]Klein KJ, Knight AP: Innovation implementation. overcoming the challenge. Curr Dir Psychol Sci 2005, 14(5):243-246.
• [4]Fixsen DL, Blasé KA, Naoom SF, Wallace F: Core implementation components. Res Soc Work Pract 2009, 19:531-540.
• [5]Grol R, Wensing M: What drives change? Barriers to and incentives for achieving evidence-based practice. The Medical Journal of Australia 2004, 180:57-60.
• [6]Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B: Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Mental Health 2009, 36:24-34.
• [7]Rogers E: Diffusion of innovation. 4th edition. Free Press, New York; 1995.
• [8]Ogden T, Forgatch M, Askeland E, Patterson GR, Bullock B: Implementation of parent management training at the national level: the case of Norway. J Soc Work Pract 2005, 19:317-329.
• [9]Ogden T, Christensen B, Sheidow A, Holth P: Bridging the gap between science and practice: the effective nationwide transport of MST programs in Norway. Journal of Children and Adolescent Substance Abuse 2008, 17:93-109.
• [10]Ogden T, Amlund-Hagen K, Askeland E, Christensen B: Implementing and evaluating evidence-based treatments of conduct problems in children and youth in Norway. Res Soc Work Pract 2009, 19:582-591.
• [11]Fixsen D, Panzano P, Naoom S, Blasé K: Measures of implementation components of the national implementation research network frameworks. Authors, Chapel Hill; 2008.
• [12]Durlak JA, DuPre EP: Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol 2008, 41:327-350.
• [13]Greenberg M, Domitovich C, Graczyk P, Zins J: The study of school-based preventive interventions: Theory, research, and practice. Center for Mental Health Services, Substance Abuse and Mental Health Administration, U.S. Department of Health and Human Services, Washington; 2001.
• [14]Biglan AA, Taylor TK: Increasing the use of science to improve child-rearing. J Prim Prev 2000, 21:207-226.
• [15]Fixsen DL, Naoom SF, Blasé KA, Friedman RM, Wallace F: [http://www.fpg.unc.edu/~nirn/resources/publications/Monograph/] webciteImplementation research: A synthesis of the literature.
• [16]Shortell SM: Increasing value: a research agenda for addressing the managerial and organizational challenges facing health care delivery in the United States. Medical Care Research and Review 2004, 61(Suppl 3):12S-30S.
• [17]Fixsen DL, Blasé K, Timbers GD, Wolf MM: In search of program Implementation: 792 replications of the teaching-family model. In Offender Rehabilitation in Practice: Implementing & Evaluating Effective Programs. Edited by Bernfield GA, Farrington DP. John Wiley & Sons, New York; 2001.
• [18]Panzano PC, Seffrin B, Chaney-Jones S, Roth D, Crane-Ross D, Massati R: The innovation diffusion and adoption research project (IDARP). In New Research in Mental Health. Edited by Roth D, Lutz W. The Ohio Department of Mental Health Office of Program Evaluation and Research, Columbus; 2004.
• [19]Fixsen DL, Blasé KA, Dua MA, Naoom SF, Van Dyke M: Implementation of evidence-based treatments for children and adolescents: Research findings and their implications for the future. In Evidence-Based Psychotherapies for Children and Adolescents. Edited by Kazdin AE, Weisz JR. The Guilford Press, New York; 2010.
• [20]Forgatch M, Patterson GR: The Oregon model of Parent Management Training (PMTO): An intervention for antisocial behavior in children and adolescents. In Evidence-Based Psychotherapies for Children and Adolescents. 2nd edition. Edited by Kazdin AE, Weisz JR. The Guilford Press, New York; 2010.
• [21]Welsh BC, Sullivan CJ, Olds DL: When early crime prevention goest to scale: a new look at the evidence. Prev Sci 2010, 11:115-125.
• [22]Henggeler SW, Schoenwald SK, Borduin CM, Rowland MD, Cunningham PB: Multisystemic Treatment of Antisocial Behaviour in Children and Adolescents. The Guilford Press, New York; 2009.
• [23]Lopes H, Calapez T: Exploring the sources and benefits of cooperation. Int J Soc Econ 2011, 38(7):607-627.
• [24]Meulman JJ, Van Der Kooij AJ, Heiser WJ: Principal Components Analysis with Nonlinear Optimal Scaling Transformations for Ordinal and Nominal Data. In Handbook of Quantitative Methods in the Social Sciences. Edited by Kaplan D. Sage Publications, Newbury Park; 2004:49-70.
• [25]Hair JF, Black WC, Babin BJ, Anderson RE, Tatham RL: Multivariate data analysis. 6th edition. Pearson Education International, New Jersey; 2006.
• [26]Linting M, Meulman JJ, Groenen PJF, Van Der Kooij JJ: Nonlinear principal component analysis: introduction and application. Psychol Methods 2007, 12:336-358.
• [27]Curran PJ, West SG, Finch JF: The robustness of test statistics to nonnormality and specification error in confirmatory factor analysis. Psychol Methods 1996, 1:16-29.
• [28]Tabachnik BG, Fidell LS: Using multivariate statistics. 4th edition. Allyn and Bacon, Needham Heights; 2001.
• [29]Mihalic SF, Irwin K: Blueprints for violence prevention: from research to real world settings. Factors influencing the successful replication of model programs. Youth violence and juvenile justice 2003, 1:1-23.
• [30]Forgatch MS, Patterson GR, DeGarmo DS: Evaluating fidelity: predictive validity for a measure of competent adherence to the Oregon model of parent management training (PMTO). Behav Ther 2005, 36:3-13.
• [31]Schoenwald SK, Hoagwood K: Effectiveness, transportability, and disseminations of interventions: what matters when? Psychiatr Serv 2001, 52:1190-1197.