Guidelines for guideline developers: a systematic review of grading systems for medical tests

Implementation Science

Mariska MG Leeflang¹  Patrick MM Bossuyt¹  Rob JPM Scholten²  Miranda W Langendam²  Gowri Gopalakrishna¹ 

[1] Department of Clinical Epidemiology, Biostatistics and Bioinformatics, Academic Medical Center, University of Amsterdam, P.O Box 22700, Amsterdam, DE, 1100, The Netherlands;Dutch Cochrane Centre. Academic Medical Center J1b-226, Postbus 22660, Amsterdam, DE, 1100, The Netherlands

关键词: Guideline development;    Grade;    Medical tests;    Diagnostic;    Quality of evidence;    Grading systems;   

Others  :  813505
DOI  :  10.1186/1748-5908-8-78

 received in 2012-10-18, accepted in 2013-06-14,  发布年份 2013

【 摘 要 】

Background

A variety of systems have been developed to grade evidence and develop recommendations based on the available evidence. However, development of guidelines for medical tests is especially challenging given the typical indirectness of the evidence; direct evidence of the effects of testing on patient important outcomes is usually absent. We compared grading systems for medical tests on how they use evidence in guideline development.

Methods

We used a systematic strategy to look for grading systems specific to medical tests in PubMed, professional guideline websites, via personal correspondence, and handsearching back references of key articles. Using the Appraisal of Guidelines for Research and Evaluation (AGREE) instrument as a starting point, we defined two sets of characteristics to describe these systems: methodological and process ones. Methodological characteristics are features relating to how evidence is gathered, appraised, and used in recommendations. Process characteristics are those relating to the guideline development process. Data were extracted in duplicate and differences resolved through discussion.

Results

Twelve grading systems could be included. All varied in the degree to which methodological and process characteristics were addressed. Having a clinical scenario, identifying the care pathway and/or developing an analytical framework, having explicit criteria for appraising and linking indirect evidence, and having explicit methodologies for translating evidence into recommendations were least frequently addressed. Five systems at most addressed these, to varying degrees of explicitness and completeness. Process wise, features most frequently addressed included involvement of relevant professional groups (8/12), external peer review of completed guidelines (9/12), and recommendations on methods for dissemination (8/12). Characteristics least often addressed were whether the system was piloted (3/12) and funder information (3/12).

Conclusions

Five systems for grading evidence about medical tests in guideline development addressed to differing degrees of explicitness the need for and appraisal of different bodies of evidence, the linking of such evidence, and its translation into recommendations. At present, no one system addressed the full complexity of gathering, assessing and linking different bodies of evidence.

【 授权许可】

   
2013 Gopalakrishna et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

【 参考文献 】

• [1]Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, Liberati A, O’Connell D, Oxman AD, Phillips B, Schünemann H, Edejer TT, Vist GE, Williams JW Jr, The GRADE Working Group: Systems for grading the quality of evidence and the strength of recommendations I: critical appraisal of existing approaches. BMC Health Serv Res 2004, 4:38. BioMed Central Full Text
• [2]West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, Lux L: Systems to rate the strength of scientific evidence. Evidence Report/Technology Assessment No. 47 (Prepared by the Research Triangle Institute-University of North Carolina Evidence-based Practice Center under Contract No. 290-97-0011). In AHRQ Publication No. 02-E016. Agency for Healthcare Research and Quality: Rockville, MD; 2002.
• [3]Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS: Evidence based medicine: what it is and what it isn’t. BMJ 1996, 13(312(7023)):71-72.
• [4]Schünemann HJ, Fretheim A, Oxman AD: Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations. Health Res Policy Syst. 2006, 5:4-21.
• [5]Bossuyt PM, Reitsma JB, Linnet K, Moons KG: Beyond diagnostic accuracy: the clinical utility of diagnostic tests. Clin Chem 2012, 58(12):1636-1643.
• [6]Feinstein AR: Misguided efforts and future challenges for research on ‘diagnostic tests. J Epidemiol Community Health 2002, 56:330-332.
• [7]Mrus JM: Getting beyond diagnostic accuracy: moving toward approaches that can be used in practice. Clin Infect Dis 2004, 38(10):1391-1393.
• [8]Guyatt GH, Oxman AD, Kunz R, Falck-Ytter Y, Vist GE, Liberati A: Schünemann HJ; GRADE working group: going from evidence to recommendations. BMJ 2008, 336(7652):1049-1051.
• [9]Bossuyt PMM, Irwig L, Craig J, Glasziou P: Comparative accuracy: assessing new tests against existing diagnostic pathways. BMJ 2006, 332:1089-1092.
• [10]Schünemann HJ, Fretheim A, Oxman AD, WHO Advisory Committee on Health Research: Improving the use of research evidence in guideline development: 1. Guidelines for guidelines. Health Res Policy Syst 2006, 21(4):13.
• [11]Beck C, Cody M, Souder E, Zhang M, Small GW: Dementia diagnostic guidelines: methodologies, results, and implementation costs. J Am Geriatr Soc 2000, 48:1195-1203.
• [12]Devillé WL, Bezemer PD, Bouter LM: Publications on diagnostic test evaluation in family medicine journals: an optimal search strategy. J Clin Epidemiol 2000, 53(1):65-69.
• [13]Leeflang MM, Scholten RJ, Rutjes AW, Reitsma JB, Bossuyt PM: Use of methodological search filters to identify diagnostic accuracy studies can lead to the omission of relevant studies. J Clin Epidemiol 2006, 59(3):234-240.
• [14]AGREE Next Steps Consortium: The AGREE II Instrument [Electronic version]. 2009. http://www.agreetrust.org webcite
• [15]Brozek JL, Akl EA, Jaeschke R, Lang DM, Bossuyt P, Glasziou P, Helfand M, Ueffing E, Alonso-Coello P, Meerpohl J, Phillips B, Horvath AR, Bousquet J, Guyatt GH: Schunemann HJ for the GRADE working group: grading quality of evidence and strength of recommendations in clinical practice guidelines: part 2 of 3. The GRADE approach to grading quality of evidence about diagnostic tests and strategies. Allergy 2009, 64:1109-1116.
• [16]Brozek JL, Akl EA, Compalati E, Kreis J, Terracciano L, Fiocchi A, Ueffing E, Andrews J, Alonso-Coello P, Meerpohl PJJ, Lang DM, Jaeschkel1 R, Williams JW Jr, Phillips B, Lethaby A, Bossuyt P, Glasziou P, Helfand M, Watine J, Afilalo M, Welch V, Montedori A, Abraha I, Horvath AR, Bousquet J, Guyatt GH, Schunemann HJ, GRADE Working Group: Grading quality of evidence and strength of recommendations in clinical practice guidelines. Part 3 of 3. The GRADE approach to developing recommendations. Allergy 2011, 66:588-595.
• [17]Schunemann HJ, Oxman AD, Brozek J, Glasziou P, Jaeschke R, Vist GE, Williams JW Jr, Kunz R, Craig J, Montori VM, Bossuyt P, Guyatt GH, GRADE Working Group: GRADE: grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ 2008, 17(336(7653):1106-1110.
• [18]Hsu J, Brozek JL, Terraciano L, Kreis J, Compalati E, Stein AT, Fiocchi A, Schünemann HJ: Application of GRADE: Making evidence-based recommendations about diagnostic tests in clinical practise guidelines. Implement Sci 2011, 6:62. BioMed Central Full Text
• [19]American Heart Association Practice Guidelines (AHAPG): Methodology Manual and Policies. From the ACCF/AHA Task Force on Practice Guidelines. Washington, D.C: American College of Cardiology Foundation; 2010. http://www.cardiosource.org/Science-And-Quality/Practice-Guidelines-and-Quality-Standards/Methodology.aspx webcite.
• [20]Dougados M, Betteridge N, Burmester GR, Euller-Ziegler L, Guillemin G, Hirvonen J, Lloyd J, Ozen S, Da Silva JAP, Emery P, Kalden JR, Kvien T, et al.: EULAR standardised operating procedures for the elaboration, evaluation, dissemination, and implementation of recommendations endorsed by the EULAR standing committees. Ann Rheum Dis 2004, 63:1172-1176.
• [21]European Society of Cardiology (ESC): Recommendations for Guideline Production. Bethesda: Committee for Practice Guidelines of the European Society of Cardiology; 2010. http://www.escardio.org/guidelines-surveys/esc-guidelines/about/Pages/rules-writing.aspx webcite.
• [22]Teutsch SM, Bradley LA, Palomaki GE, Haddow JE, Piper M, Calonge N, Dotson WD, Douglas MP, Berg AO: The Evaluation of Genomic Applications in Practice and Prevention (EGAPP) initiative: methods of the EGAPP Working Group. Genet Med 2009, 11(1):3-14.
• [23]Australian Government National Health and Medical Research Council: NHMRC levels of evidence and grades for recommendations for developers of guidelines. 2009. http://www.nhmrc.gov.au/guidelines/how-nhmrc-develops-its-guidelines webcite
• [24]Australian Government National Health and Medical Research Council: How to review the evidence: systematic identification and review of the scientific literature. Handbook series on preparing clinical practice guidelines. Canberra: National Health and Medical Research Council; 1999. http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp65.pdf webcite.
• [25]NHMRC National Health and Medical Research Council: A guide to the development, implementation and evaluation of clinical practice guidelines. Canberra: National Health and Medical Research Council; 1998. http://www.nhmrc.gov.au/_files_nhmrc/publications/attachments/cp30.pdf webcite.
• [26]National Institute for Health and Clinical Excellence (NICE): Centre for health technology evaluation. Diagnostics Assessment Programme (DAP). Programme manual. 2011. http://www.nice.org.uk/media/A0B!/97/DAPManualFINAL.pdf webcite.
• [27]U.S. Preventative Services Task Force: Procedure Manual. AHRQ Publication No. 08-05118-EF. 2008. http://www.uspreventiveservicestaskforce.org/uspstf08/methods/procmanual.pdf webcite.
• [28]Institute for Clinical Systems Improvement. https://www.icsi.org/guidelines__more/about_icsi_guidelines/ webcite.
• [29]Ebell MH, Siwek J, Weiss BD, Woolf SH, Susman J, Ewigman B, Bowman M: Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician 2004, 69(3):548-556.
• [30]Scottish Intercollegiate Guidelines Network: SIGN Guideline Development Handbook: SIGN 50.  . http://www.sign.ac.uk/methodology/index.html webcite
• [31]Oxford Centre for Evidence-based Medicine: Levels of Evidence. 2009. http://www.cebm.net/index.aspx?o=1025 webcite
• [32]Thornton J, Alderson P, Tan T, Turner C, Latchem S, Shaw E, Ruiz F, Reken S, Mugglestone MA, Hill J, Neilson J, Westby M, et al.: Introducing GRADE across the NICE clinical guideline program. J Clin Epidemiol 2013, 66(2):124-131.
• [33]Cuello-García CA, Dávalos-Rodríguez ML: Perceptions and attitudes towards different grading systems from clinical guidelines developers. J Eval Clin Pract 2009, 15:1074-1076.
• [34]Guyatt GH, Oxman AD, Vist G, Kunz R, Falck-Ytter Y, Alonso-Coello P: Schünemann HJ for the GRADE Working Group (2008): Rating quality of evidence and strength of recommendations GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008, 336:924-926.
• [35]Burgers JS, Cluzeau FA, Hanna SE, Hunt C, Grol R: Characteristics of high-quality guidelines: evaluation of 86 clinical guidelines developed in ten European countries and Canada. Int J Technol Assess Healthcare 2003, 19(1):148-157.
• [36]Grilli R, Magrini N, Penna A, Mura G, Liberati A: Practice guidelines developed by specialty societies: the need for a critical appraisal. Lancet 2000, 355:103-105.
• [37]Centre for Evidence based Medicine. University of Oxford;  . http://www.cebm.net/index.aspx?o=1036 webcite
• [38]Ferrante di Ruffano L, Hyde CJ, McCaffery KJ, Bossuyt PMM, Deeks JJ: Assessing the value of diagnostic tests - a framework for designing and evaluating trials. BMJ 2012, 344:e686.
• [39]Fryback DG, Thornbury JR: The efficacy of diagnostic imaging. Med Decis Making 1991, 11:88-94.