Reporting standards for guideline-based performance measures

Implementation Science

Ina Kopp¹  Markus Follmann⁶  Raija Sipilä⁴  Patrice Lindsay⁵  Beth Shaw³  Tim Stokes²  Monika Nothacker¹ 

[1] Arbeitsgemeinschaft der Wissenschaftlichen Medizinischen Fachgesellschaften—Institut für Medizinisches Wissensmanagement (AWMF-IMWI), Philipps-Universität Marburg, Karl-von-Frisch-Str.1, Marburg 35043, Germany;Department of General Practice and Rural Health, Dunedin School of Medicine, University of Otago, Dunedin, New Zealand;National Institute for Health and Care Excellence (NICE), Manchester, UK;The Finnish Medical Society Duodecim, Current Care Guidelines, Helsinki, Finland;Heart and Stroke Foundation, Ottawa, Canada;German Cancer Society, Berlin, Germany

关键词: Delphi technique;    Process assessment (health care);    Reporting standard;    Quality indicator;    Performance measure;    Guideline adherence;    Guideline;   

Others  :  1235411
DOI  :  10.1186/s13012-015-0369-z

 received in 2015-09-25, accepted in 2015-12-30,  发布年份 2016

【 摘 要 】

Background

The Guidelines International Network (G-I-N) aims to promote high quality clinical guideline development and implementation. Guideline-based performance measures are a key implementation tool and are widely used internationally for quality improvement, quality assurance, and pay for performance in health care. There is, however, no international consensus on best methods for guideline-based performance measures. In order to address this issue, the G-I-N Performance Measures Working Group aimed to develop a set of consensus-based reporting standards for guideline-based performance measure development and re-evaluation.

Methods

Methodology publications on guideline-based performance measures were identified from a systematic literature review and analyzed. Core criteria for the development and evaluation process of guideline-based performance measures were determined and refined into draft standards with an associated rationale and description of the evidence base. In a two-round Delphi-process, the group members appraised and approved the draft standards. After the first round, the group met to discuss comments and revised the drafts accordingly.

Results

Twenty-one methodology publications were reviewed. The group reached strong consensus on nine reporting standards concerning: (1) selection of clinical guidelines, (2) extraction of clinical guideline recommendations, (3) description of the measure development process, (4) measure appraisal, (5) measure specification, (6) description of the intended use of the measure, (7) measure testing/validating, (8) measure review/re-evaluation, and (9) composition of the measure development panel.

Conclusions

These proposed international reporting standards address core components of guideline-based performance measure development and re-evaluation. They are intended to contribute to international reporting harmonization and improvement of methods for performance measures. Further research is required regarding validity, acceptability, and practicality.

【 授权许可】

   
2016 Nothacker et al.

【 图 表 】

Fig. 1.

Fig. 2.

【 参考文献 】

• [1]Institute of Medicine, Committee on Standards for Developing Trustworthy Clinical Practice Guidelines. Clinical Practice Guidelines We Can Trust 2011. Available from: http://www.iom.edu/Reports/2011/Clinical-Practice-Guidelines-We-Can-Trust.aspx. [Accessed: 20.07.2015]
• [2]Crossing the quality chasm: a New Health System for the 21st Century. National Academy Press, Wahington D.C; 2001.
• [3]Donabedian A. Evaluating the quality of medical care. The Milbank Memorial Fund Quarterly Milbank MemFund Q. 1966;44:166–206.
• [4]Kopp IB. Von Leitlinien zur Qualitätssicherung. Bundesgesundheitsblatt Gesundheitsforschung Gesundheitsschutz. 2011; 54(2):160-5.
• [5]Nix M, Nothacker M, Stokes T. Guideline derived performance measures: what do guideline and performance measure developers need from each other?(Abstract) G-I-N Conference. German Medical Science GMS Publishing House, Berlin; 2012.
• [6]Wilson C. Hospital-wide quality assurance: models for implementation and development. W.B. Saunders, London; 1987.
• [7]Van Engen-Verheul M, Kemps H, Kraaijenhagen R, De Keizer N, Peek N. Modified Rand method to derive quality indicators: a case study in cardiac rehabilitation. Stud Health Technol Inform. 2011; 169:88-92.
• [8]De Koning JSA, Klazinga NS. Appraisal of Indicators through Research and Evaluation (AIRE). Orde van Medisch Specialisten, Utrecht; 2006.
• [9]Reiter A, Fischer B, Kötting J, Geraedts M, Jäckel WH, Döbler K. QUALIFY—a tool for assessing quality indicators. Z Arztl Fortbild Qualitatssich. 2007; 101(10):683-8.
• [10]National Quality Measures Clearing House (NQMC). Tutorial of attributes; 2013. Available from: http://www.qualitymeasures.ahrq.gov/tutorial/attributes.aspx. [Accessed 20.07.2015]
• [11]Center for Health Policy/Center for Primary Care and Outcomes Research & Battelle Memorial Institute. Quality indicator measure development, implementation, maintenance, and retirement (prepared by Battelle, under contract no. 290-04-0020). Rockville, MD: Agency for Healthcare Research and Quality; 2011. Available from: .[Accessed 20.07.2015].
• [12]Gagliardi AR, Brouwers MC, Palda VA, Lemieux-Charles L, Grimshaw JM. How can we improve guideline use? A conceptual framework of implementability. Implement Sci. 2011; 6:26. BioMed Central Full Text
• [13]Appraisal of Guidelines for Research and Evaluation (AGREE), Agree Trust. APPRAISAL OF GUIDELINES FOR RESEARCH & EVALUATION II; 2009. Available from: http://www.agreetrust.org/. [Accessed 20.07.2015]
• [14]Kötter T, Blozik E, Scherer M. Methods for the guideline-based development of quality indicators—a systematic review. Implement Sci. 2012; 7:21. BioMed Central Full Text
• [15]Blozik E, Nothacker M, Bunk T, Szecsenyi J, Ollenschläger G, Scherer M. Simultaneous development of guidelines and quality indicators—how do guideline groups act? A worldwide survey. Int J Health Care Qual Assur. 2012; 25(8):712-29.
• [16]Board of Trustees, G-I-N. Guidelines International Network Strategic Plan 2010–2013. Available from: http://www.g-i-n.net/document-store/g-i-n-strategy-2010-10-13.pdf. [Accessed 20.07.2015].
• [17]Shea BJ, Hamel C, Wells GA, Bouter LM, Kristjansson E, Grimshaw J et al.. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews. J Clin Epidemiol. 2009; 62(10):1013-20.
• [18]Murphy MK, Black NA, Lamping DL, McKee CM, Sanderson CFB, Askham J et al.. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998; 2(3):1-87.
• [19]Using clinical practice guidelines to evaluate the quality of care. US Dept. of Health and Human Sciences, Rockville; 1995.
• [20]AHCPR. Designing and implementing guidelines-based performance measures. Qual Lett Healthc Lead [Internet]. 1995; (7):[21–3 pp.].
• [21]Baker R, Fraser RC. Development of review criteria: linking guidelines and assessment of quality. BMJ. 1995; 311(7001):370-3.
• [22]Campbell SM, Braspenning J, Hutchinson A, Marshall M. Research methods used in developing and applying quality indicators in primary care. Qual Saf Health Care. 2002; 11(4):358-64.
• [23]AQUA-Institut. Allgemeine Methoden im Rahmen der sektorenübergreifenden Qualitätssicherung im Gesundheitswesen nach §137a SGB V. Version 2.0: Göttingen, 2010.
• [24]Ärztliches Zentrum für Qualität in der Medizin (Ed.)Manual Qualitätsindikatoren. Berlin: äzq Schriftenreihe Band 36; 2009. Available from: http://www.aezq.de/aezq/publikationen/schriftenreihe. [Accessed 20.07.2015]
• [25]Bergman DA. Evidence-based guidelines and critical pathways for quality improvement. Pediatrics. 1999; 103(1 Suppl E):225-32.
• [26]Califf RM, Peterson ED, Gibbons RJ, Garson A, Brindis RG, Beller GA et al.. Integrating quality into the cycle of therapeutic development. J Am Coll Cardiol. 2002; 40(11):1895-901.
• [27]Graham WJ. Criterion-based clinical audit in obstetrics: bridging the quality gap? Best Pract Res Clin Obstet Gynaecol. 2009; 23(3):375-88.
• [28]Spertus JA, Eagle KA, Krumholz HM, Mitchell KR, Normand SL et al.. American College of Cardiology and American Heart Association methodology for the selection and creation of performance measures for quantifying the quality of cardiovascular care. Circulation. 2005; 111(13):1703-12.
• [29]Wollersheim H, Hermens R, Hulscher M, Braspenning J, Ouwens M, Schouten J et al.. Clinical indicators: development and applications. Neth J Med. 2007; 65(1):15-22.
• [30]Hutchinson A, McIntosh A, Anderson J, Gilbert C, Field R. Developing primary care review criteria from evidence-based guidelines: coronary heart disease as a model. Br J Gen Pract. 2003; 53(494):690-6.
• [31]Advani A, Goldstein M, Shahar Y, Musen MA. Developing quality indicators and auditing protocols from formal guideline models: knowledge representation and transformations. AMIA Annu Symp Proc. 2003;11–5.
• [32]Duffy FF, Narrow W, West JC, Fochtmann LJ, Kahn DA, Suppes T et al.. Quality of care measures for the treatment of bipolar disorder. Psychiatr Q. 2005; 76(3):213-30.
• [33]Golden WE, Hermann RC, Jewell M, Brewster C. Development of evidence-based performance measures for bipolar disorder: overview of methodology. J Psychiatr Pract. 2008; 14 Suppl 2:18-30.
• [34]LaClair BJ, Reker DM, Duncan PW, Horner RD, Hoenig H. Stroke care: a method for measuring compliance with AHCPR guidelines. Am J Phys Med Rehabil. 2001; 80(3):235-42.
• [35]AQUA-Institut. Allgemeine Methoden im Rahmen der sektorenübergreifenden Qualitätssicherung im Gesundheitswesen nach §137a SGB V. Version 3.0: Göttingen, 2013.
• [36]Campbell SM, Kontopantelis E, Hannon K, Burke M, Barber A, Lester HE. Framework and indicator testing protocol for developing and piloting quality indicators for the UK quality and outcomes framework. BMC Fam Pract. 2011; 12:85. BioMed Central Full Text
• [37]Spertus JA, Bonow RO, Chan P, Diamond GA, Drozda JP, Kaul S et al.. ACCF/AHA new insights into the methodology of performance measurement: a report of the American College of Cardiology Foundation/American Heart Association Task Force on performance measures. J Am Coll Cardiol. 2010; 56(21):1767-82.
• [38]Follmann M KI, Klinkhammer-Schalke M, Pottkämper K, Wesselmann S, Wöckel A, Albert U.S. Updating Guideline based Quality Indicators. The Methodology of the German Breast Cancer Guideline Development Group. gms [Abstract]. 2012. Available from: http://www.egms.de/static/de/meetings/gin2012/12gin048.shtml. [Accessed 20.07.2015]
• [39]Qaseem A, Forland F, Macbeth F, Ollenschläger G, Phillips S, van der Wees P et al.. Guidelines International Network: toward international standards for clinical practice guidelines. Ann Intern Med. 2012; 156(7):525-31.
• [40]Fervers B, Burgers JS, Haugh MC, Latreille J, Mlika-Cabanne N, Paquet L et al.. Adaptation of clinical guidelines: literature review and proposition for a framework and procedure. Int J Qual Health Care. 2006; 18(3):167-76.
• [41]Shekelle P, Woolf S, Grimshaw JM, Schünemann HJ, Eccles MP. Developing clinical practice guidelines: reviewing, reporting, and publishing guidelines; updating guidelines; and the emerging issues of enhancing guideline implementability and accounting for comorbid conditions in guideline development. Implement Sci. 2012; 7:62. BioMed Central Full Text
• [42]Nothacker MJ, Langer T, Weinbrenner S. Quality indicators for National Disease Management Guidelines using the example of the National Disease Management Guideline for “Chronic Heart Failure”. Z Evid Fortbild Qual Gesundhwes. 2011; 105(1):27-37.
• [43]Andrews JC, Schünemann HJ, Oxman AD, Pottie K, Meerpohl JJ, Coello PA et al.. GRADE guidelines: 15. Going from evidence to recommendation-determinants of a recommendation’s direction and strength. J Clin Epidemiol. 2013; 66(7):726-35.
• [44]Baker DW, Qaseem A, Reynolds PP, Gardner LA, Schneider EC. Design and use of performance measures to decrease low-value services and achieve cost-conscious care. Ann Intern Med. 2013; 158(1):55-9.
• [45]National Quality Measures Clearing House (NQMC). Tutorial on quality measures. Desirable Attributes of a measure.2011. Available from: URL: http://www.qualitymeasures.ahrq.gov/tutorial/index.aspx.[Accessed 20.07.2015}
• [46]National Quality Measures Clearing House (NQMC). Uses of Quality Measures 2013. Available from: http://www.qualitymeasures.ahrq.gov/tutorial/using.aspx.[Accessed 20.07.2015]
• [47]Wallwiener M, Brucker SY, Wallwiener D. Multidisciplinary breast centres in Germany: a review and update of quality assurance through benchmarking and certification. Arch Gynecol Obstet. 2012; 285(6):1671-83.
• [48]Doran T, Kontopantelis E, Valderas JM, Campbell S, Roland M, Salisbury C et al.. Effect of financial incentives on incentivised and non-incentivised clinical activities: longitudinal analysis of data from the UK Quality and Outcomes Framework. BMJ. 2011; 342:d3590.
• [49]Kruse GB, Polsky D, Stuart EA, Werner RM. The impact of hospital pay-for-performance on hospital and Medicare costs. Health Serv Res. 2012; 47(6):2118-36.
• [50]Raleigh VS, Foot C. Getting the measure of Quality—opportunities and challenges London: The King’s Fund; 2010. Available from: http://www.kingsfund.org.uk/publications/getting-measure-quality [Accessed 20.07.2015]
• [51]Mainz J. Defining and classifying clinical indicators for quality improvement. Int J Qual Health Care. 2003; 15(6):523-30.
• [52]Freeman T. Using performance indicators to improve health care quality in the public sector: a review of the literature. Health Serv Manage Res. 2002; 15(2):126-37.
• [53]McDonald R, Roland M. Pay for performance in primary care in England and California: comparison of unintended consequences. Ann Fam Med. 2009; 7(2):121-7.
• [54]Powell AA, White KM, Partin MR, Halek K, Christianson JB, Neil B et al.. Unintended consequences of implementing a national performance measurement system into local practice. J Gen Intern Med. 2012; 27(4):405-12.
• [55]Agency for Healthcare Research and Quality. Quality indicator measure development, implementation, maintenance, and retirement: Rockville; 2011.
• [56]Shaw EJ, Sutcliffe D, Lacey T, Stokes T. Assessing depression severity using the UK Quality and Outcomes Framework depression indicators: a systematic review. The National Academic Press, Washington, D.C. Br J Gen Pract. 2013;63(610):e309–17.
• [57]De Bruin-Kooistra M, Amelink-Verburg MP, Buitendijk SE, Westert GP. Finding the right indicators for assessing quality midwifery care. Int J Qual Health Care. 2012; 24(3):301-10.
• [58]Reeves D, Doran T, Valderas JM, Kontopantelis E, Trueman P, Sutton M et al.. How to identify when a performance indicator has run its course. BMJ. 2010; 340:c1717.
• [59]De Koning J. Development and validation of a measurement instrument for appraising indicator quality: appraisal of indicators through research and evaluation (AIRE) instrument. German Medical Science [Internet]. 2007; 2007/07. Available from: http://www.egms.de/static/de/meetings/gmds2007/07gmds798.shtml. [Accessed 20.07.2015]
• [60]Kötter T, Schaefer FA, Scherer M, Blozik E. Involving patients in quality indicator development—a systematic review. Patient Prefer Adherence. 2013; 7:259-68.
• [61]German Medical Association (GMA), National Association of Statutory Health Insurance Physicians (NASHIP), Association of the Scientific Medical Societies (AWMF). National Programm for Disease Management Guidelines. Method Report 2010. Available from: Available from: http://www.leitlinien.de/nvl/methodik/. [Accessed 20.07.2015]
• [62]Healthcare Improvement Scotland . SIGN 50: A guidelines developpers handbook 2011. Available from: http://www.sign.ac.uk/pdf/sign50.pdf. [Accessed20.07.2015]
• [63]Venkatesh A, Goodrich K, Conway PH. Opportunities for quality measurement to improve the value of care for patients with multiple chronic conditions. Ann Intern Med. 2014; 161(10 Suppl):S76-80.
• [64]Weldring T, Smith SM. Patient-Reported Outcomes (PROs) and Patient-Reported Outcome Measures (PROMs). Health Serv Insights. 2013; 6:61-8.
• [65]Fischer F, Krämer A, Klose K. Relevance of quality of life as a quality indicator in guideline development. Gesundheitswesen. 2015; 77(5):e119-32.
• [66]Anderson JL, Heidenreich PA, Barnett PG, Creager MA, Fonarow GC, Gibbons RJ et al.. ACC/AHA statement on cost/value methodology in clinical practice guidelines and performance measures: a report of the American College of Cardiology/American Heart Association Task Force on Performance Measures and Task Force on Practice Guidelines. J Am Coll Cardiol. 2014; 63(21):2304-22.
• [67]von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP, et al. The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies. Lancet. 2007;370(9596):1453–7.