Instrumentation issues in implementation science

Implementation Science

Bryan J Weiner¹  Cara C Lewis³  Ruben G Martinez² 

[1] 1102-C McGavran-Greenberg Hall, University of North Carolina at Chapel Hill, 135 Dauer Drive, Chapel Hill 27599-7411, USA;Department of Psychology, Virginia Commonwealth University, 806 West Franklin St., Richmond 23220, VA, USA;Department of Psychiatry & Behavioral Sciences, University of Washington, 1959 NE Pacific Street, Seattle 98195-6560, WA, USA

关键词: Mixed methods;    Psychometrics;    Dissemination;    Implementation;    Instruments;   

Others  :  1146603
DOI  :  10.1186/s13012-014-0118-8

 received in 2013-11-19, accepted in 2014-08-26,  发布年份 2014

【 摘 要 】

Background

Like many new fields, implementation science has become vulnerable to instrumentation issues that potentially threaten the strength of the developing knowledge base. For instance, many implementation studies report findings based on instruments that do not have established psychometric properties. This article aims to review six pressing instrumentation issues, discuss the impact of these issues on the field, and provide practical recommendations.

Discussion

This debate centers on the impact of the following instrumentation issues: use of frameworks, theories, and models; role of psychometric properties; use of `home-grown¿ and adapted instruments; choosing the most appropriate evaluation method and approach; practicality; and need for decision-making tools. Practical recommendations include: use of consensus definitions for key implementation constructs; reporting standards (e.g., regarding psychometrics, instrument adaptation); when to use multiple forms of observation and mixed methods; and accessing instrument repositories and decision aid tools.

Summary

This debate provides an overview of six key instrumentation issues and offers several courses of action to limit the impact of these issues on the field. With careful attention to these issues, the field of implementation science can potentially move forward at the rapid pace that is respectfully demanded by community stakeholders.

【 授权许可】

   
2014 Martinez et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Siegel AE: Sidney Siegel: a memoir. In Decision and Choice, Contributions of Sidney Siegel. McGraw-Hill, New York, NY; 1964.
• [2]Proctor EK, Landsverk J, Aarons G, Chambers D, Glisson C, Mittman B: Implementation research in mental health services: an emerging science with conceptual, methodological, and training challenges. Adm Policy Ment Health 2009, 36:24-34.
• [3]Cook DA, Beckman TJ: Current concepts in validity and reliability for psychometric instruments: theory and application. Am J Med 2006, 119:166.e17-166.e16.
• [4]Downing SM, Haladyna TM: Test item development: validity evidence from quality assurance procedures. Appl Meas Educ 1997, 10:61-82.
• [5]Proctor EK, Brownson RC: Measurement Issues in Dissemination and Implementation Research. In Dissemination and Implementation Research in Health: Translating Science to Practice. Edited by Brownson R, Colditz G, Proctor E. Oxford University Press, Inc, New York, NY; 2012.
• [6]Grimshaw J, Eccles M, Thomas R, MacLennan G, Ramsay C, Fraser C, Vale L: Toward evidence based quality improvement. J Gen Intern Med 2006, 21:S14-S20.
• [7]Achenbach TM: Advancing assessment of children and adolescents: commentary on evidence-based assessment of child and adolescent disorders. J Clin Child Adolesc 2005, 34:541-547.
• [8]Hunsley J, Mash EJ: Evidence-based assessment. The Oxford handbook of clinical psychology 2011, 76-97.
• [9]Michie S, Fixsen D, Grimshaw JM, Eccles MP: Specifying and reporting complex behaviour change interventions: the need for a scientific method. Implement Sci 2009, 4:40. BioMed Central Full Text
• [10]Michie S, Johnston M, Francis J, Hardeman W, Eccles M: From theory to intervention: mapping theoretically derived behavioural determinants to behaviour change techniques. Appl Psychol 2008, 57:660-680.
• [11]Rabin BA, Brownson RC, Haire-Joshu D, Kreuter MW, Weaver NL: A glossary for dissemination and implementation research in health. J Public Health Manag Pract 2008, 14:117-123.
• [12]Tabak RG, Khoong EC, Chambers DA, Brownson RC: Bridging research and practice: models for dissemination and implementation research. Am J Prev Med 2012, 43:337-350.
• [13]Weinert CR, Mann HJ: The science of implementation: changing the practice of critical care. Curr Opin Crit Care 2008, 14:460-465.
• [14]Aarons GA, Hurlburt M, Horwitz SM: Advancing a conceptual model of evidence-based practice implementation in public service sectors. Adm Policy Ment Health 2011, 38:4-23.
• [15]Damschroder LJ, Aron DC, Keith RE, Kirsh SR, Alexander JA, Lowery JC: Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science. Implement Sci 2009, 4:50. BioMed Central Full Text
• [16]Messick S: Validity of psychological assessment: validation of inferences from persons' responses and performances as scientific inquiry into score meaning. Am Psychol 1995, 50:741-749.
• [17]Proctor EK, Silmere H, Raghavan R, Hovmand P, Aarons GA, Bunger A, Griffey R, Hensley M: Outcomes for implementation research: conceptual distinctions, measurement challenges, and research agenda. Adm Policy Ment Health 2011, 38:65-76.
• [18]Chaudoir SR, Dugan AG, Barr CHI: Measuring factors affecting implementation of health innovations: a systematic review of structural, organizational, provider, patient, and innovation level measures. Implement Sci 2013, 8:22. BioMed Central Full Text
• [19][http://www.britannica.com/EBchecked/topic/1192819/wiki] webcite Encyclopedia Britannica. In .
• [20][http://cfirwiki.net/wiki/index.php?title=Main_Page] webcite Consolidated Framework for Implementation Research (CFIR) Wiki. In .
• [21]May C: Towards a general theory of implementation. Implement Sci 2013, 8:18. BioMed Central Full Text
• [22]Stichter J, Conroy MA: A critical analysis of the role of measurement on the validity of emotional and behavioral disorders (EBD) research. Behav Disorders 2004, 30:7-18.
• [23]Weiner BJ: A theory of organizational readiness for change. Implement Sci 2009, 4:67. BioMed Central Full Text
• [24]Chor KHB, Wisdom JP, Olin S-CS, Hoagwood KE, Horwitz SM: Measures for predictors of innovation adoption.Adm Policy Ment Health 2014, 1¿29.
• [25]Association AER: American Psychological Association, National Council on Measurement in Education: The standards for educational and psychological testing. American Psychological Association, Washington, DC; 1999.
• [26]Streiner DL: Starting at the beginning: an introduction to coefficient alpha and internal consistency. J Pers Assess 2003, 80:99-103.
• [27]Walsh WB, Betz NE: Tests and assessment. Prentice-Hall Inc, Englewood Cliffs, NJ; 1995.
• [28]Eyde LD, Robertson GJ, Krug SE: Responsible Test Use: Case Studies for Assessing Human Behavior. American Psychological Association, Washington, DC; 1995.
• [29]Osipow SH: Developing instruments for use in counseling. J Couns Dev 1991, 70:322-326.
• [30]Kendler KS: The dappled nature of causes of psychiatric illness: replacing the organic¿functional/hardware¿software dichotomy with empirically based pluralism. Mol Psychiatry 2012, 17:377-388.
• [31]Glisson C, Landsverk J, Schoenwald S, Kelleher K, Hoagwood KE, Mayberg S, Green P: Assessing the organizational social context (OSC) of mental health services: Implications for research and practice. Adm Policy Ment Health 2008, 35:98-113.
• [32]Martinez RG, Lewis CC: Instrumentation Issues in Implementation Science. In Seattle, WA: Presented at Seattle Implementation Research Conference; 2013.
• [33]Hinkin TR: A brief tutorial on the development of measures for use in survey questionnaires. Organ Res Methods 1998, 1:104-121.
• [34]DeVellis RF: Scale development: Theory and applications. Sage Publications, Thousand Oaks, CA; 2011.
• [35]Collins D: Pretesting survey instruments: an overview of cognitive methods. Qual Life Res 2003, 12:229-238.
• [36]Palinkas LA, Aarons GA, Horwitz S, Chamberlain P, Hurlburt M, Landsverk J: Mixed method designs in implementation research. Adm Policy Ment Health 2011, 38:44-53.
• [37]Squires JE, Hutchinson AM, Boström A-M, O'Rourke HM, Cobban SJ, Estabrooks CA: To what extent do nurses use research in clinical practice? A systematic review. Implement Sci 2011, 6:21. BioMed Central Full Text
• [38]Podsakoff PM, MacKenzie SB, Lee J-Y, Podsakoff NP: Common method biases in behavioral research: a critical review of the literature and recommended remedies. J Appl Psychol 2003, 88:879-903.
• [39]Hofmann W, Gawronski B, Gschwendner T, Le H, Schmitt M: A meta-analysis on the correlation between the Implicit Association Test and explicit self-report measures. Pers Soc Psychol Bull 2005, 31:1369-1385.
• [40]Greenwald AG, Poehlman TA, Uhlmann EL, Banaji MR: Understanding and using the Implicit Association Test: III. Meta-analysis of predictive validity. J Pers Soc Psychol 2009, 97:17.
• [41]Martino S, Ball S, Nich C, Frankforter TL, Carroll KM: Correspondence of motivational enhancement treatment integrity ratings among therapists, supervisors, and observers. Psychother Res 2009, 19:181-193.
• [42]Kimberly J, Cook JM: Organizational measurement and the implementation of innovations in mental health services. Adm Policy Ment Health 2008, 35:11-20.
• [43]Dattilio FM, Edwards DJA, Fishman DB: Case studies within a mixed methods paradigm: toward a resolution of the alienation between researcher and practitioner in psychotherapy research. Psychother Theor Res Pract Train 2010, 47:427-441.
• [44]Gaglio B, Glasgow RE: Evaluation approaches for dissemination and implementation research. In Dissemination and implementation research in health: translating science to practice. Edited by Brownsone RC, Colditz GA, Proctor EK. Oxford University Press, New York, NY; 2012:327-356.
• [45]McLeod BD, Jensen-Doss A, Ollendick TH: Diagnostic and behavioral assessment in children and adolescents: a clinical guide. The Guilford Press, New York, NY; 2013.
• [46]Stange KC, Zyzanski SJ, Smith TF, Kelly R, Langa DM, Flocke SA, Jaén CR: How valid are medical records and patient questionnaires for physician profiling and health services research?: A comparison with direct observation of patient visits. Med Care 1998, 36:851-867.
• [47]English DJ, Brandford CC, Coghlan L: Data-based organizational change: the use of administrative data to improve child welfare programs and policy. Child Welfare 2000, 79:499-515.
• [48]Powell BJ, McMillen JC, Proctor EK, Carpenter CR, Griffey RT, Bunger AC, Glass JE, York JL: A compilation of strategies for implementing clinical innovations in health and mental health. Med Care Res Rev 2012, 69:123-157.
• [49]Glasgow RE, Riley WT: Pragmatic measures: what they are and why we need them. Am J Prev Med 2013, 45:237-243.
• [50][http:/ / www.societyforimplementationresearc hcollaboration.org/ sirc-projects/ sirc-instrument-project/ ] webcite Society for Implementation Research Collaboration Instrument Review Project. In .
• [51][http://www.gem-beta.org] webcite National Cancer Institute Grid-Enabled Measures Project. In .
• [52]Aarons GA: Mental health provider attitudes toward adoption of evidence-based practice: the Evidence-Based Practice Attitude Scale (EBPAS). Ment Health Serv Res 2004, 6:61-74.
• [53]Rogers EM: Diffusion of innovations. Simon and Schuster, New York, NY; 2003.
• [54]Kramer DM, Cole DC: Sustained, intensive engagement to promote health and safety knowledge transfer to and utilization by workplaces. Sci Commun 2003, 25:56-82.
• [55]Jacobson N, Butterill D, Goering P: Development of a framework for knowledge translation: understanding user context. J Health Serv Res Pol 2003, 8:94-99.
• [56]Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O: Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Q 2004, 82:581-629.
• [57]Dodson EA, Brownson RC, Weiss SW: Policy dissemination research. In Dissemination and implementation research in health: translating science to practice. Edited by Brownson RC, Colditz GA, Proctor EK. Oxford University Press, New York, NY; 2012:437-458.
• [58]Atun R, de Jongh T, Secci F, Ohiri K, Adeyi O: Integration of targeted health interventions into health systems: a conceptual framework for analysis. Health Policy Plan 2010, 25:104-111.
• [59]Glasgow RE, Vogt TM, Boles SM: Evaluating the public health impact of health promotion interventions: the RE-AIM framework. Am J Public Health 1999, 89:1322-1327.
• [60]Wandersman A, Duffy J, Flaspohler P, Noonan R, Lubell K, Stillman L, Blachman M, Dunville R, Saul J: Bridging the gap between prevention research and practice: the interactive systems framework for dissemination and implementation. Am J Community Psychol 2008, 41:171-181.
• [61]Farkas M, Anthony WA: Bridging science to service: using Rehabilitation Research and Training Center program to ensure that research-based knowledge makes a difference. J Rehabil Res Dev 2007, 44:879-892.
• [62]Damush T, Bravata D, Plue L, Woodward-Hagg H, William L: Facilitation of Best Practices (FAB) Framework: Stroke QUERI Center Annual Report. Am J Prev Med 2008, 43:337-350.
• [63]Feldstein AC, Glasgow RE: A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf 2008, 34:228-243.
• [64]Bowen S, Zwi AB: Pathways to `evidence-informed¿ policy and practice: a framework for action. PLoS Med 2005, 2:e166.
• [65]Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F: Implementation research: A synthesis of the literature. University of South Florida, Louis de la Parte Florida Mental Health Institute, The National Implementation Research Network, Tampa, FL; 2005.
• [66]Glisson C, Schoenwald SK: The ARC organizational and community intervention strategy for implementing evidence-based children's mental health treatments. Ment Health Serv Res 2005, 7:243-259.
• [67]Kitson A, Harvey G, McCormack B: Enabling the implementation of evidence based practice: a conceptual framework. Qual Health Care 1998, 7:149-158.
• [68]Torrey WC, Lynde DW, Gorman P: Promoting the implementation of practices that are supported by research: The National Implementing Evidence-Based Practice Project. Child Adolesc Psychiatr Clin N Am 2005, 14:297-306.