Protocol for a statewide randomized controlled trial to compare three training models for implementing an evidence-based treatment

Implementation Science

Stanley Mrozowski²  Mark Chaffin³  Satish Iyengar⁴  Shelley A. Hiegel¹  Kristen F. Schaffner¹  Sarah Taber-Thomas⁵  Ashley T. Scudder⁵  David J. Kolko⁵  Amy D. Herschell⁶ 

[1] University of Pittsburgh Medical Center, Pittsburgh, PA, USA;Pennsylvania Office of Mental Health and Substance Abuse Services, Harrisburg, PA, USA;Georgia State University School of Public Health, Atlanta, GA, USA;University of Pittsburgh Department of Statistics, Pittsburgh, PA, USA;University of Pittsburgh School of Medicine, Pittsburgh, PA, USA;Western Psychiatric Institute and Clinic, University of Pittsburgh School of Medicine, Pittsburgh, PA, USA

关键词: Parent–Child Interaction Therapy;    Evidence-based treatment;    Distance education;    Train-the-trainer;    Cascading model;    Learning collaborative;    Therapist training;    Implementation;   

Others  :  1229129
DOI  :  10.1186/s13012-015-0324-z

 received in 2015-07-01, accepted in 2015-09-10,  发布年份 2015

【 摘 要 】

Background

Evidence-based treatments (EBTs) are available for treating childhood behavioral health challenges. Despite EBTs’ potential to help children and families, they have primarily remained in university settings. Little empirical evidence exists regarding how specific, commonly used training and quality control models are effective in changing practice, achieving full implementation, and supporting positive client outcomes.

Methods/design

This study (NIMH RO1 MH095750; ClinicalTrials.gov Identifier: NCT02543359), which is currently in progress, will evaluate the effectiveness of three training models (Learning Collaborative (LC), Cascading Model (CM), and Distance Education (DE)) to implement a well-established EBT , Parent-Child Interaction Therapy, in real-world, community settings. The three models differ in their costs, skill training, quality control methods, and capacity to address broader implementation challenges. The project is guided by three specific aims: (1) to build knowledge about training outcomes, (2) to build knowledge about implementation outcomes, and (3) to test the differential impact of training clinicians using LC, CM, and DE models on key client outcomes. Fifty (50) licensed psychiatric clinics across Pennsylvania were randomized to one of the three training conditions: (1) LC, (2) CM, or (3) DE. The impact of training on practice skills (clinician level) and implementation/sustainment outcomes (clinic level) are being evaluated at four timepoints coinciding with the training schedule: baseline, 6 (mid), 12 (post), and 24 months (1 year follow-up). Immediately after training begins, parent–child dyads (client level) are recruited from the caseloads of participating clinicians. Client outcomes are being assessed at four timepoints (pre-treatment, 1, 6, and 12 months after the pre-treatment).

Discussion

This proposal builds on an ongoing initiative to implement an EBT statewide. A team of diverse stakeholders including state policy makers, payers, consumers, service providers, and academics from different, but complementary areas (e.g., public health, social work, psychiatry), has been assembled to guide the research plan by incorporating input from multidimensional perspective.

Trial registration

ClinicalTrials.gov: NCT02543359

【 授权许可】

   
2015 Herschell et al.

【 图 表 】

Fig. 1.

【 参考文献 】

• [1]Lavigne JV, Gibbons RD, Christoffel KK, Rosenbaum D, Binns H, Dawson N et al.. Prevalence rates and correlates of psychiatric disorders among preschool children. J Am Acad Child Adolesc Psychiatry. 1996; 35(2):204-214.
• [2]Tremblay RE, Pihl RO, Vitaro F, Dobkin PL. Predicting early onset of male antisocial behavior from preschool behavior. Arch Gen Psychiatry. 1994; 51(9):732-736.
• [3]Block J, Block JH, Keyes S. Longitudinally foretelling drug usage in adolescence: early childhood personality and environmental precursors. Child Dev. 1988; 59(2):336-355.
• [4]Caspi A, Moffitt TE, Newman DL, Silva PA. Behavioral observations at age 3 years predict adult psychiatric disorders. Arch Gen Psychiatry. 1996; 53:1033-1039.
• [5]Egger H. Toddler with temper tantrums: A careful assessment of a dysregulated preschool child. In: DSM-IV-TR casebook and treatment guide for child mental health. Galanter CA, Jensen P, editors. Am Psychiatr US, Arlington, VA, US; 2009: p.365-384.
• [6]Campbell SB. Behavior problems in preschool children: a review of recent research. J Child Psychol Psychiatry. 1995; 36:113-149.
• [7]Kessler RC, Berglund P, Demler O, Jin R, Merikangas KR, Walers RR. Lifetime prevalence and age-of-onset distributions of DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005; 62:593-603.
• [8]Kazdin AE. Conduct disorders in childhood and adolescence. 2nd ed. Sage, Thousand Oaks; 1995.
• [9]Schumann E, Durning P, Eyberg SM, Boggs SR. Screening for conduct problem behavior in pediatric settings using the Eyberg Child Behavior Inventory. Ambul Child Health. 1996; 2:35-41.
• [10]Brestan EV, Eyberg SM. Effective pscyhosocial treatments of conduct-disordered children and adolescents: 29 years, 82 studies, and 5,272 kids. J Clin Psychol. 1998; 27(2):180-189.
• [11]Eyberg SM. Members of the Child Study Laboratory. Parent–child Interaction Therapy: integrity checklists and session materials. 1999.
• [12]Herschell AD, Calzada EJ, Eyberg SM, McNeil CB. Parent–child interaction therapy: new directions in research. Cogn Behav Pract. 2002; 9:9-16.
• [13]Gallagher N. Effects of Parent–child Interaction Therapy on young children with disruptive behavior disorders. Bridges Practice-Based Res Syntheses. 2003; 1:1-17.
• [14]Thomas R, Simmer-Gembeck MJ. Behavioral outcomes of Parent–child Interaction Therapy and Triple P-Positive Parenting Program: a review and meta-analysis. J Abnorm Child Psychol. 2007; 35:475-495.
• [15]Eisenstadt TH, Eyberg SM, McNeil CB, Newcomb K, Funderburk B. Parent–child interaction therapy with behavior problem children: relative effectiveness of two stages and overall treatment outcome. J Clin Child Psychol. 1993; 22:42-51.
• [16]McNeil CB, Clemens-Mowrer L, Gurwitch RH, Funderbunk BW. Assessment of a new procedure to prevent timeout escape in preschoolers. Child Fam Behav Ther. 1994; 16(3):27-35.
• [17]Schuhmann E, Foote R, Eyberg SM, Boggs SR, Algina J. Parent–child interaction therapy: interim report of a randomized trial with short-term maintenance. J Clin Child Psychol. 1998; 27:34-45.
• [18]Boggs SR, Eyberg SM, Edwards DL, Rayfield A, Jacobs J, Bagner D et al.. Outcomes of Parent–child Interaction Therapy: a comparison of treatment completers and study dropouts one to three years later. Child Fam Behav Ther. 2004; 26:1-22.
• [19]Eyberg SM, Funderbunk BW, Hembree-Kigin T, McNeil CB, Querido JG, Hood KK. Parent–child interaction therapy with behavior problem children: one and two year maintenance of treatment effects in the family. Child Fam Behav Ther. 2001; 23:1-20.
• [20]Funderburk B, Eyberg SM, Newcomb K, McNeil CB, Hembree-Kigin T, Capage L. Parent–child Interaction Therapy with behavior problem children: maintenance of treatment effects in the school setting. Child Fam Behav Ther. 1998; 20:17-38.
• [21]Hood KK, Eyberg SM. Outcomes of Parent–child Interaction Therapy: mothers’ reports of maintenance three to six years after treatment. J Clin Child Adolesc Psychol. 2003; 32(3):419-429.
• [22]Chaffin M, Silovsky JF, Funderburk B, Valle LA, Brestan EV, Balachova T et al.. Parent–child interaction therapy with physically abusive parents: efficacy for reducing future abuse reports. J Consult Clin Psychol. 2004; 72:500-510.
• [23]Lanier P, Kohl PL, Benz J, Swinger D, Mousette P, Drake B. Parent–child interaction therapy in a community setting: examining outcomes, attrition, and treatment setting. Res Soc Work Pract. 2011; 24(1):689-698.
• [24]Lyon AR, Budd KS. A community mental health implementation of Parent–child interaction therapy. J Child Fam Stud. 2010; 19:654-668.
• [25]A report of the Surgeon General. Author, Rockville, MD; 2009.
• [26]President’s New Freedom Commission on Mental Health. Report of the President’s New Freedom Commission on Mental Health. 2004.
• [27]McHugh RK, Barlow DH. The dissemination and implementation of evidence-based psychological treatments: a review of the current efforts. Am Psychol. 2010; 65(2):73-84.
• [28]Herschell AD, Kolko DJ, Baumann BL, Davis AC. The role of therapist training in the implementation of psychosocial treatments: a review and critique with recommendations. Clin Psychol Rev. 2010; 30:448-466.
• [29]The breakthrough series: IHI’s collaborative model for achieveing breakthrough improvement in innovation series. Institute for Healthcare Improvement, Boston; 2003.
• [30]Kilo CM. A framework for collaborative improvement: lessons from the Institute for Healthcare Improvement’s breakthrough series. Qual Manag Health Care. 1998; 6:1-13.
• [31]Markiewicz J, Ebert L, Ling D, Amaya-Jackson L, Kisiel C. Learning collaborative toolkit. Los Angeles, CA and Durham, NC, National Center for Child Traumatic Stress; 2006.
• [32]Flamm BL, Berwick DM, Kabcenall AI. Reducing cesarean section rates safely: lessons learned from a “breakthrough series” collaborative. Birth. 1998; 25:117-124.
• [33]Lendon BE, Wilson IB, McInnes K, Landrum B, Hirschhon L, Marsden PV et al.. Effects of a quality improvement collaborative on the outcome care of patients with HIV infection: the EQHIV study. Ann Intern Med. 2004; 140:887-896.
• [34]Glasgow RE, Funnell MM, Bonomi AE, Davis C, Beckham V, Wagner EH. Self-management aspects of the improving chronic illness care breakthrough series: Implementation with diabetes and heart failure. Ann Behav Med. 2002; 24:80-87.
• [35]Cretin S, Shortell SM, Keller EB. An evaluation of collaborative interventions to improve chronic illness care: framework and study design. Eval Rev. 2004; 28(1):28-51.
• [36]Young PC, Glade GB, Stoddard GJ, Norlin C. Evaluation of a learning collaborative to improve the delivery of preventive services by pediatric practices. Pediatrics. 2006; 117(5):1469-1479.
• [37]Ebert L, Amaya-Jackson L, Markiewicz J, Kisiel C, Fairbank J. Use of the breakthrough series collaborative to support broad and sustained use of evidence-based trauma treatment for children in community practice settings. Adm Policy Ment Health. 2011:1–13. doi:10.1007/s10488-011-0347-y.
• [38]Cavaleri MA, Gopalan G, McKay M, Appel A, Bannon WM, Bigley MF et al.. Impact of a learning collaborative to improve child mental health service use among low-income urban youth and families. Best Pract Ment Health. 2006; 2(2):67-79.
• [39]Cavaleri MA, Franco LM, McKay M, Appel A, Bannon WM, Bigley MF et al.. The sustainability of a learning collaborative to improve mental health service use among low-income urban youth and families. Best Pract Ment Health. 2007; 3(2):52-61.
• [40]Cavaleri MA, Perez M, Burton G, Penn M, Beharie N, Hoagwood KE. Developing the Support, Teambuilding, and Referral (STAR) Intervention: a research/community partnership. Child Adolesc Ment Health. 2010; 15(1):56-59.
• [41]Rutkowski BA, Gallon S, Rawson RA, Freese TE, Bruehl A, Crevecoeur-MacPhail D et al.. Improving client engagement and retention in treatment: The Los Angeles County experience. J Subst Abuse Treat. 2010; 39:78-86.
• [42]Solberg LI, Kottke TE, Brekke ML, Magnan S, Davidson G, Calomeni CA et al.. Failure of a continuous quality improvement intervention to increase the delivery of preventative services: a randomized trial. Eff Clin Pract. 2000; 3(3):105-115.
• [43]Rogers ES, Cohen BF, Danley KS, Hutschinson D, Anthony WA. Training mental health workers in psychiatric rehabilitation. Schizophr Bull. 1986; 12(4):709-719.
• [44]Chamberlain P, Price J, Reid J, Landsverk J. Cascading implementation of a foster and kinship parent intervention. Child Welfare. 2008; 87(5):27.
• [45]Hein D, Litt LC, Cohen L, Miele GM, Campbell A. Preparing, training, and integrating staff in providing integrated treatment. In: Trauma services for women in substance abuse: an integrated approach. American Psychological Association, Washington D.C; 2009: p.143-161.
• [46]Martino S, Brigham GS, Higgins C, Gallon S, Freese TE, Albright LM et al.. Partnerships and pathways of dissemination: the NIDA-SAMHSA Blending Initiative in the Clinical Trials Network. J Subst Abuse Treat. 2010; 38 Suppl 1:S31-S43.
• [47]Coogle CL. The families who care project: meeting educational needs of African American and rural family caregivers dealing with dementia. Educ Gerontol. 2002; 28(1):59-71.
• [48]Gabel LL, Pearsol JA. The twin epidemics of substance use and HIV: a state-level response using a train-the-trainer model. Fam Pract. 1993; 10(4):400-405.
• [49]Jason LA, Lapp C, Kenney KK, Lupton T. An innovative approach in training health care workers to diagnose and manage patients with CFS. In: Advances in pyschology research. Alexandra AM, editor. Nova Science Publishers, Hauppauge, NY; 2008: p.273-281.
• [50]Nyamathi A, Vatsa M, Khakha DC, McNeese-Smith D, Leake B, Fahey JL. HIV knowledge improvement among nurses in India: using a train-the-trainer program. JANAC. 2008; 19(6):443-449.
• [51]Pullum JD, Sanddal ND, Obbink K. Training for rural prehospital providers: a retrospective analysis from Montana. Prehosp Emerg Care. 1999; 3(3):231-238.
• [52]Tibbles LR, Smith AE, Manzi SC. Train-the-trainer for hospital-wide safety training. J Nurs Staff Dev. 1993; 9(6):266-269.
• [53]Virani R, Malloy P, Ferrell BR, Kelly K. Statewide efforts in promoting palliative care. J Palliat Med. 2008; 11(7):991-996.
• [54]Booth-Kewley S, Gilman PA, Shaffer RA, Brodine SK. Evaluation of sexually transmitted disease/human immunodeficiency virus prevention train-the-trainer program. Mil Med. 2001; 166(4):304-310.
• [55]Coogle CL, Osgood NJ, Parham IA. A statewide model detection and prevention program for geriatric alcoholism and alcohol abuse: increased knowledge among service providers. Community Ment Health J. 2000; 36(2):137-148.
• [56]Gadomski AM, Wolff D, Tripp M, Lewis C, Short LM. Changes in health care providers’ knowledge, attitudes, beliefs, and behaviors regarding domestic violence, following a multifaceted intervention. Acad Med. 2001; 76(10):1045-1052.
• [57]Moon RY, Calabrese T, Aird L. Reducing the risk of sudden infanct death syndrome in child care and changing provider practices: lessons learned from a demonstration project. Pediatrics. 2008; 122(4):788-798.
• [58]Tziraki C, Graubard BI, Manley M, Kosary C, Moler JE, Edwards BK. Effect of training on adoption of cancer prevention nutrition-related activities by primary care practices: results of a randomized, controlled study. JGIM. 2000; 15(3):155-162.
• [59]Well M, Stokols D, McMahan S, Clitheroe C. Evaluation of a worksite injury and illness prevention program: do the effects of the REACH OUT training program reach the employees? J Occup Health Psychol. 1997; 2(1):25-34.
• [60]Zapata LB, Coreil J, Entrekin N. Evaluation of triple touch: an assessment of program delivery. Cancer Pract. 2001; 9 Suppl 1:S23-S30.
• [61]Demchak M, Browder DM. An evaluation of the pyramid model of staff training in group homes for adults with severe handicaps. Educ Train Ment Retard. 1990; 25(2):150-163.
• [62]Shore BA, Iwata BA, Vollmer TR, Lerman DC, Zarcone JR. Pyramidal staff training in the extension of treatment for severe behavior disorders. J Appl Behav Anal. 1995; 28(3):323-330.
• [63]Anderson SE, Youngson SC. Introduction of a child sexual abuse policy in a health district. Child Soc. 1990; 4:401-419.
• [64]Miller WR, Sorenson JL, Selzer JA, Brigham GS. Disseminating evidence-based practices in substance abuse treatment: a review with suggestions. J Subst Abuse Treat. 2006; 31(1):25-39.
• [65]Worrall JM, Fruzzetti A. Improving peer supervision ratings of therapist performance in dialectical behavior therapy: an internet-based training system. Psychother Theory Res Pract Train. 2009; 46(4):476-479.
• [66]Sholomskas DE, Syracuse-Siewert G, Rounsaville BJ, Ball SA, Nuro KF. We don’t train in vain: a dissemination trial of three strategies of training clinicians in cognitive behavioral therapy. J Consult Clin Psychol. 2005; 73(1):106-115.
• [67]TF-CBT web: first year report. Medical University of South Carolina, Charleston, SC; 2007.
• [68]Suda KT, Miltenberger RG. Evaluation of staff management strategies to increase positive interactions in a vocational setting. Behav Res Treat. 1993; 8(2):69-88.
• [69]Miller WR, Yahne CE, Moyers TB, Martinez J, Pirritano M. A randomized trial of methods to help clinicians learn motivational learning. J Consult Clin Psychol. 2004; 72(6):1050-1062.
• [70]Sanders MR, Turner KMT, Markie-Dadds C. The development and dissemination of the Triple P-Positive Parenting Program: a multilevel, evidence-based system of parenting and family support. Prev Sci. 2002; 3:173-189.
• [71]Ford JK, Weissbein DA. Training of transfer: an updated review. Perform Improv Q. 1997; 10:22-41.
• [72]Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B. Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health. 2009; 36:24-34.
• [73]Baldwin T, Ford JK. Transfer of training: a review and directions for future research. Pers Psychol. 1988; 41:63-105.
• [74]Kraiger K, Ford JD, Salas E. Application of cognitive, skill-based, and affective theories of learning outcomes to new methods of training evaluation. J Appl Psychol. 1993; 78:311-328.
• [75]Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A et al. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health. 2010. doi:10.1007/s10488-010-0319-7.
• [76]Herschell AD, Kogan JN, Celedonia KL, Lindhiem O, Stein BD. Evaluation of an implementation initiative for embedding dialectical behavior therapy in community settings. Eval Program Plann. 2014; 43:55-63.
• [77]Kolko DJ, Baumann BL, Herschell AD, Hart JA, Holden EA, Wisniewski SR. Implementation of AF-CBT with community practitioners serving child welfare and mental health: a randomized trial. Child Maltreat. in press.
• [78]Rollins AL, Salyers MP, Tsai J, Lydick JM. Staff turnover in a statewide implementation of ACT: Relationship with ACT fidelity and other team characteristics. Admin Policy Mental Health Mental Health Serv Res. 2010; 37:417-426.
• [79]Woltmann EM, Whitley R, McHugo GJ, Brunette M, Torrey WC, Coots L et al.. The role of staff turnover in the implementation of evidence-based practices in mental health care. Psychiatr Serv. 2008; 59(7):732-737.
• [80]Southam-Gerow MA, Weisz JR, Chu BC, McLeod BD, Gordis EB, Connor-Smith JK. Does cognitive behavioral therapy for youth anxiety outperform usual care in community clinics? An initial effectiveness test. J Am Acad Child Adolesc Psychiatry. 2010; 49(10):1043-1052.
• [81]Weisz JR, Gordis EB, Chu BC, McLeod BD, Updegraff A, Southam-Gerow MA et al.. Cognitive-behavioral therapy versus usual clinical care for youth depression: an Initial test of transportability to community clinics and clinicians. J Consult Clin Psychol. 2009; 77(3):383-396.
• [82]Chamberlain P, Brown CH, Saldana L, Reid J, Wand W, Marsenich L et al.. Engaging and recruiting counties in an experiment on implementing evidence-based practice in California. Adm Policy Ment Health. 2008; 35:250-260.
• [83]Aarons GA, Wells RS, Zagursky K, Fettes DL, Palinkas LA. Implementing evidence-based practice in community mental health agencies: a multiple stakeholder analysis. Am J Public Health. 2009; 99(11):2087-2095.
• [84]Wang W, Saldana L, Brown CH, Chamberlain P. Factors that influenced county system leaders to implement an evidence-based program: a baseline survey within a randomized controlled trial. Implement Sci. 2010; 5:72.
• [85]Eyberg SM, Funderburk B, McNeil CB, Niec L, Urquiza AJ, Zebbell NM. Training guidelines for parent–child interaction therapy. 2009.
• [86]TF-CBT web: a web-based learning course for trauma-focused cognitive-behavioral therapy. http://tfcbt.musc.edu (2005) Assessed 30 March 2015.
• [87]Proctor E, Silmere H, Raghavan R, Hovmand P, Aarons G, Bunger A et al.. Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Admin Policy Mental Health Mental Health Serv Res. 2011; 38(2):65-76.
• [88]Baron RM, Kenny DM. The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. J Pers Soc Psychol. 1986; 51:1173-1182.
• [89]Kraemer HC et al.. Mediators and moderators of treatment effects in randomized clinical trials. Arch Gen Psychiatry. 2002; 59:877-883.
• [90]Little R, Rubin D. Statistical analysis with missing data. John Wiley & Sons, Inc. Hoboken,NJ: 2002.
• [91]Cohen J. Statistical power analysis for the behavioral sciences. Lawrence Erlbaum Associates, Inc, Hillsdale, NJ; 1998.
• [92]Henggeler SW, Schoenwald SK, Liao JG, Letourneau EJ, Edwards DL. Transporting efficacious treatments to field settings: the link between supervisory practices and therapist fidelity in MST programs. J Clin Child Adolesc Psychol. 2002; 31(2):155-167.
• [93]Beidas RS, Kendall PC. Training therapists in evidence-based practice: a critical review of studies from a systems-contextual perspective. Clin Psychol Sci Pract. 2010; 17(1):1-30.
• [94]Martino S, Ball S, Nich C, Canning-Ball M, Rounsaville B, Carroll K. Teaching community program clinicians motivational interviewing using expert and train-the-trainer strategies. Addiction. 2010; 106(2):428-441.
• [95]Scudder AB, Herschell AD. Building an evidence-base for the training of evidence-based treatments: use of an expert informed approach. Child Youth Serv Rev. 2015; 55:84-92.
• [96]Funderburk B, Nelson M. PCIT Coaches Quiz. 2014.
• [97]Funderburk B, Gurwitch R, Shanley J, Chase R, Nelson M, Bard E et al.. FIRST Coach Coding System for CDI Coaching Manual. 2014.
• [98]Kolko DJ. Treatment of child physical abuse: an effectiveness trial. 2012.
• [99]Herschell AD, McNeil CB, Urquiza AJ, McGrath JM, Zebbell NM, Timmer SG et al.. Evaluation of a treatment manual and workshops for disseminating, Parent–child Interaction Therapy. Admin Policy Mental Health Mental Health Serv Res. 2009; 36:63-81.
• [100]Chafouleas SM, Briesch AM, Riley-Tillman TC, McCoach DB. Moving beyond assessment of treatment acceptability: an examination of the factor structure of the Usage Rating Profile—Intervention (URP-I). Sch Psychol Q. 2009; 24:36-47.
• [101]Addis ME, Krasnow AD. A national survey of practicing psychologists’ attitudes toward psychotherapy treatment manuals. J Consulting Clin Psychol. 2000; 68(2):331-339.
• [102]Schoenwald SK et al.. A survey of the infrastructure for children’s mental health services: implications for the implementation of empirically supported treatments. Adm Policy Ment Health. 2008; 35:84-97.
• [103]Hoagwood K, Schoenwald SK, Chapman JE. Dimensions of organizational readiness—revised (DOOR-R). Unpublished Instrument. 2003
• [104]Lehman WE, Greener JM, Simpson DD. Assessing organizational readiness for change. J Subst Abuse Treat. 2002; 22(4):197-209.
• [105]Kolko DJ, Herschell AD, Baumann BL. Treatment Implementation Feedback Form. Unpublished Manuscript. Pittsburgh PA: University of Pittsburgh School of Medicine. 2006.
• [106]Kolko DJ, Dorn LD, Burkstein OG, Pardini D, Holden EA, Hart JD. Community vs. clinic-based modular treatment of children with early-onset ODD or CD: a clinical trial with three-year follow-up. J Abnorm Child Psychol. 2009; 37:591-609.
• [107]Eyberg S, Funderburk B. Parent–Child Interaction Therapy protocol. PCIT International, Inc. Gainsville, FL: 2011.
• [108]Herschell AD. Translating an evidence-based treatment into community practice: implementing parent–child interaction therapy in an urban domestic violence shelter. 2011.
• [109]Olmstead T, Carroll K, Canning-Ball M, Martino S. Cost and cost-effectiveness of three strategies for training clinicians in motivational interviewing. in press.
• [110]Aarons GA. Evidence-based practice implementation: the impact of public versus private sector organization type on organizational support, provider attitudes, and adoption of evidence-based practice. Impl Sci. 2009; 4:83.
• [111]Eyberg SM, Pincus D. Eyberg Child Behavior Inventory and Sutter-Eyberg Behavior Inventory: revised professional manual. Psychological Assessment Resources, Inc., Odessa, FL; 2000.
• [112]Wolraich ML, Lambert W, Doffing MA, Bickman L, Simmons T, Worley K. Psychometric properties of the Vanderbilt ADHD diagnostic parent rating scale in a referred population. J Pediatr Psychol. 2003; 28(8):559-568.
• [113]Frick PJ, Hare RD. Antisocial process screening device: APSD. Multi-Health Systems, Toronto; 2001.
• [114]Kroenke K, Spitzer RL, Williams JB. The Phq‐9. JGIM. 2001; 16(9):606-613.
• [115]Spitzer RL, Kroenke K, Williams JB, Löwe B. A brief measure for assessing generalized anxiety disorder: the GAD-7. Arch Intern Med. 2006; 166(10):1092-1097.
• [116]Elgar FJ, Waschbusch DA, Dadds MR, Sigvaldason N. Development and validation of a short form of the Alabama Parenting Questionnaire. J Child Fam Stud. 2007; 16(2):243-259.
• [117]Kazdin AE, Holland L, Crowley M, Breton S. Barriers to treatment participation scale: evaluation and validation in the context of child outpatient treatment. J Child Psychol Psychiatry. 1997; 38(8):1051-1062.
• [118]Brestan EV, Jacobs JR, Rayfield AD, Eyberg SM. A consumer satisfaction measure for parent–child treatments and its relation to measures of child behavior change. Behav Ther. 2000; 30(1):17-30.
• [119]Gyamfi P, Walrath C, Burns BJ, Stephens RL, Geng Y, Stambaugh L. Family education and support services in systems of care. J Emotion Behav Disord. 2009; 18(1):14-26.
• [120]Ben-Dror R. Employee turnover in community mental health organizations: A developmental stages study. Community Ment Health J. 1994;30:14.
• [121]Sheidow AJ, Schoenwald SK, Wagner HR, Allred CA, & Burns BJ. Predictors of workforce turnover in a tranposted treatment program. Adm Policy Ment Health and Ment Health Serv Res. 2007;34(1):45–56.