Grading quality of evidence and strength of recommendations

David Atkins

Dana Best

Peter A Briss

Martin Eccles

Margaret C Haugh+19 authors

Correspondence to: Andrew D Oxman, Informed Choice Research Department, Norwegian Health Services Research Centre, PO Box 7004, St Olavs Plass, 0130 Oslo, Norway on.enilno@namxo

Accepted 2004 Mar 5.

Short abstract

Clinical guidelines are only as good as the evidence and judgments they are based on. The GRADE approach aims to make it easier for users to assess the judgments behind recommendations

Short abstract

Abstract

Users of clinical practice guidelines and other recommendations need to know how much confidence they can place in the recommendations. Systematic and explicit methods of making judgments can reduce errors and improve communication. We have developed a system for grading the quality of evidence and the strength of recommendations that can be applied across a wide range of interventions and contexts. In this article we present a summary of our approach from the perspective of a guideline user. Judgments about the strength of a recommendation require consideration of the balance between benefits and harms, the quality of the evidence, translation of the evidence into specific circumstances, and the certainty of the baseline risk. It is also important to consider costs (resource utilisation) before making a recommendation. Inconsistencies among systems for grading the quality of evidence and the strength of recommendations reduce their potential to facilitate critical appraisal and improve communication of these judgments. Our system for guiding these complex judgments balances the need for simplicity with the need for full and transparent consideration of all important issues.

Abstract

WMD = weighted mean difference, RRR = relative risk reduction.

Notes

Members of GRADE Working group are listed at the end of this article

Members of the Grades of Recommendation Assessment, Development and Evaluation (GRADE) Working Group who have contributed to this article include David Atkins, Dana Best, Peter A Briss, Martin Eccles, Yngve Falck-Ytter, Signe Flottorp, Gordon H Guyatt, Robin T Harbour, Margaret C Haugh, David Henry, Suzanne Hill, Roman Jaeschke, Gillian Leng, Alessandro Liberati, Nicola Magrini, James Mason, Philippa Middleton, Jacek Mrukowicz, Dianne O'Connell, Andrew D Oxman, Bob Phillips, Holger J Schünemann, Tessa Tan-Torres Edejer, Helena Varonen, Gunn E Vist, John W Williams Jr, Stephanie Zaza.

The National Institute for Clinical Excellence (NICE) for England and Wales and the Polish Institute for Evidence-Based Medicine (PIEBM) have provided support for meetings of the GRADE Working Group. The institutions with which members of the Working Group are affiliated have provided intramural support. Alessandro Liberati's participation in GRADE activities was supported by a grant from the Ministero Università e Ricerca Scientifica (M.I.U.R., Progetto COFIN 2001).

Contributors: All of the members of the GRADE Working Group listed above have contributed to the preparation of this manuscript and the development of the ideas contained in it, participated in at least one meeting, and read and commented on drafts of this article. GHG and ADO led the process. GEV has had primary responsibility for preparing the evidence profiles used in the pilot study and coordinating the process.

Competing interests: Most of the members of the GRADE Working Group have a vested interest in another system of grading the quality of evidence and the strength of recommendations.

Notes

Members of GRADE Working group are listed at the end of this article

Competing interests: Most of the members of the GRADE Working Group have a vested interest in another system of grading the quality of evidence and the strength of recommendations.

References

1. Canadian Task Force on the Periodic Health ExaminationThe periodic health examination. CMAJ 1979;121: 1193-254. [Google Scholar]
2. Sackett DLRules of evidence and clinical recommendations on the use of antithrombotic agents. Chest 1986;89(suppl 2): 2-3S. [[PubMed][Google Scholar]
3. Sackett DLRules of evidence and clinical recommendations on the use of antithrombotic agents. Arch Intern Med 1986:146: 464-5. [PubMed][Google Scholar]
4. Sackett DLRules of evidence and clinical recommendations on the use of antithrombotic agents. Chest 1989;95: 2-4S. [[PubMed][Google Scholar]
5. Cook DJ, Guyatt GH, Laupacis A, Sackett DL. Rules of evidence and clinical recommendations on the use of antithrombotic agents. Antithrombotic therapy consensus conference. Chest 1992;102(suppl 4): 305-11S. [[PubMed]
6. US Department of Health and Human Services, Public Health Service, Agency Health Care Policy and Research. Acute pain management: operative or medical procedures and trauma. Rockville, MD: Agency for Health Care Policy and Research Publications, 1992. (AHCPR Pub 92-0038.)
7. Gyorkos TW, Tannenbaum TN, Abrahamowicz M, Oxman AD, Scott EA, Millson ME, et alAn approach to the development of practice guidelines for community health interventions. Can J Public Health 1994;85(suppl 1): S8-13. [[PubMed][Google Scholar]
8. Hadorn DC, Baker DDevelopment of the AHCPR-sponsored heart failure guideline: methodologic and procedural issues. Jt Comm J Qual Improv 1994;20: 539-54. [[PubMed][Google Scholar]
9. Cook DJ, Guyatt GH, Laupacis A, Sackett DL, Goldberg RJClinical recommendations using levels of evidence for antithrombotic agents. Chest 1995;108(suppl 4): 227-30S. [[PubMed][Google Scholar]
10. Guyatt GH, Sackett DL, Sinclair JC, Hayward R, Cook DJ, Cook RJ, et alUsers' guide to the medical literature IX: a method for grading health care recommendations. JAMA 1995;274: 1800-4. [[PubMed][Google Scholar]
11. Scottish Intercollegiate Guidelines Network (SIGN). Forming guideline recommendations. In: A guideline developers' handbook. Edinburgh: SIGN, 2001. (Publication No 50.) (accessed 8 Feb 2004).[PubMed]
12. US Preventive Services Task Force. Guide to clinical preventive services. 2nd ed. Baltimore: Williams and Wilkins, 1996: xxxix-lv.
13. Eccles M, Clapp Z, Grimshaw J, Adams PC, Higgins B, Purves I, et alNorth of England evidence based guidelines development project: methods of guideline development. BMJ 1996;312: 760-2. [Google Scholar]
14. Centro per la Valutazione della Efficacia della Assistenza Sanitaria (CeVEAS).Schema di grading CeVEAS. (accessed 18 May 2004).[PubMed]
15. Guyatt G, Schünemann H, Cook D, Jaeschke R, Pauker S, Bucher HGrades of recommendation for antithrombotic agents. Chest 2001;119:3S-7S. (accessed 8 Feb 2004). [[PubMed]
16. Phillips B, Ball C, Sackett D, Badenoch D, Straus S, Haynes B, Dawes M. Levels of evidence and grades of recommendations. Oxford: Oxford Centre for Evidence-Based Medicine. (accessed 8 Feb 2004).[PubMed]
17. National Health and Medical Research Council. How to use the evidence: assessment and application of scientific evidence. Canberra: AusInfo, 2000. (accessed 8 Feb 2004).[PubMed]
18. Harbour R, Miller JA new system for grading recommendations in evidence based guidelines. BMJ 2001;323: 334-6. [Google Scholar]
19. Roman SH, Silberzweig SB, Siu ALGrading the evidence for diabetes performance measures. Eff Clin Pract 2000;3: 85-91. [[PubMed][Google Scholar]
20. Woloshin SArguing about grades. Eff Clin Pract 2000;3: 94-5. [[PubMed][Google Scholar]
21. Guyatt GH, Schünemann H, Cook D, Pauker S, Sinclair J, Bucher H, et alGrades of recommendation for antithrombotic agents. Chest 2001;119: 3-7S. [[PubMed][Google Scholar]
22. Atkins D, Best D, Shapiro EN, edsThird US Preventive Services Task Force: background, methods and first recommendations. Am J Prev Med 2001;20: 3(suppl):1- 108. [PubMed][Google Scholar]
23. Woolf SH, Atkins DThe evolving role of prevention in health care: contributions of the US Preventive Services Task Force. Am J Prev Med 2001;20: 3(suppl):13-20. [[PubMed][Google Scholar]
24. Harris RP, Helfand M, Woolf SH, Lohr KN, Mulrow CD, Teutsch SM, et alCurrent methods of the US Preventive Services Task Force: a review of the process. Am J Prev Med 2001;20:3 (suppl): 21-35. [[PubMed][Google Scholar]
25. Briss PA, Zaza S, Pappaioanou M, Fielding J, Wright-De Aguero L, et alDeveloping an evidence-based guide to community preventive services—methods. Am J Prev Med 2000;18 (suppl 1): 35-43. [[PubMed][Google Scholar]
26. Zaza S, Wright-De A, Briss PA, Truman BI, Hopkins DP, Hennessy MH, et alData collection instrument and procedure for systematic reviews in the guide to community preventive services. Am J Prev Med 2000;18(suppl 1): 44-74. [[PubMed][Google Scholar]
27. Greer N, Mosser G, Logan G, Halaas GWA practical approach to evidence grading. Jt Comm J Qual Improv 2000;26: 700-12. [[PubMed][Google Scholar]
28. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, et al. Systems to rate the strength of scientific evidence. Rockville, MD: Agency for Healthcare Research and Quality, 2002: 64-88. (AHRQ publication No 02-E016.)
29. Guyatt G, Drummond R, eds. Users' guide to the medical literature. Chicago, IL: AMA Press, 2002: 55-154.
30. Clarke M, Oxman AD, eds. Assessment of study quality. Cochrane reviewers' handbook 4.1.5 section 6. In: Cochrane Library. Issue 4. Oxford: Update Software, 2002.
31. Jüni P, Altman DG, Egger M. Assessing the quality of randomised controlled trials. In: Egger M, Davey Smith G, Altman DG, eds. Systematic reviews in health care: meta-analysis in context. London: BMJ Books, 2001: 87-121.
32. West S, King V, Carey TS, Lohr KN, McKoy N, Sutton SF, et al. Systems to rate the strength of scientific evidence. Rockville, MD: Agency for Healthcare Research and Quality, 2002: 51-63. (AHRQ publication No 02-E016.)
33. Kunz R, Vist G, Oxman AD. Randomisation to protect against selection bias in health-care trials (Cochrane methodology review). In: Cochrane Library Issue 4. Oxford: Update Software, 2002.
34. Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, et alComparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA 2001;286: 821-30. [[PubMed][Google Scholar]
35. Kleijnen J, Gøtzsche P, Kunz RA, Oxman AD, Chalmers ISo what's so special about randomisation? In: Chalmers I, Maynard A, eds. Non-random reflections on health care research: on the 25th anniversary of Archie Cochrane's effectiveness and efficiency. London: BMJ, 1997: 93-106.[Google Scholar]
36. Lacchetti C, Guyatt G. Surprising results of randomized controlled trials. In: Guyatt G, Drummond R, eds. Users' guide to the medical literature. Chicago, IL: AMA Press, 2002: 247-65.
37. Hulley S, Grady D, Bush T, Furberg C, Herrington D, Riggs B, et alRandomized trial of estrogen plus progestin for secondary prevention of coronary heart disease in postmenopausal women. JAMA 1998;280: 605-13. [[PubMed][Google Scholar]
38. Writing Group for the Women's Health Initiative Investigators. Risks and benefits of estrogen plus progestin in healthy postmenopausal women. Principal results from the women's health initiative randomized controlled trial. JAMA 2002;288: 321-33. [[PubMed]
39. Dans A, McAlister F, Dans L, Richardson WS, Straus S, Guyatt G. Applying results in individual patients. In: Guyatt G, Drummond R, eds. Users' guide to the medical literature. Chicago, IL: AMA Press, 2002: 369-84.
40. McAlister F, Laupacis A, Wells G. Drug class effects. In: Guyatt G, Drummond R, eds. Users' guide to the medical literature. Chicago, IL: AMA Press, 2002: 415-31.
41. Echt DS, Liebson PR, Mitchell LB, Peters RW, Obias-Manno D, Barker AH, et al. Mortality and morbidity in patients receiving encainide, flecainide, or placebo. The cardiac arrhythmia suppression trial. N Engl J Med 1991;324: 781-8. [[PubMed]
42. Riggs BL, Hodgson SF, O'Fallon WM, Chao EY, Wahner HW, Muhs JM, et alEffect of fluoride treatment on the fracture rate in postmenopausal women with osteoporosis. N Engl J Med 1990;322: 802-9. [[PubMed][Google Scholar]
43. Worster A, Preyra I, Weaver B, Haines TThe accuracy of noncontrast helical computed tomography versus intravenous pyelography in the diagnosis of suspected acute urolithiasis: a meta-analysis. Ann Emerg Med 2002;40: 280-6. [[PubMed][Google Scholar]
44. Worster A, Haines TDoes replacing intravenous pyelography with noncontrast helical computed tomography benefit patients with suspected acute urolithiasis? Can Assoc Radiol J 2002;53: 241. [[PubMed][Google Scholar]
45. Song F, Altman DG, Glenny AM, Deeks JJValidity of indirect comparison for estimating efficacy of competing interventions: evidence from published meta-analyses. BMJ 2003;326: 472. [Google Scholar]
46. Bross IDJPertinency of an extraneous variable. J Chron Dis 1967;20: 487-95. [[PubMed][Google Scholar]
47. Thompson DC, Rivara FP, Thompson RHelmets for preventing head and facial injuries in bicyclists. Cochrane Database Syst Rev 2000;(2): CD001855. [[PubMed][Google Scholar]
48. Devereaux PJ, Choi PT, Lacchetti C, Weaver B, Schünemann HJ, Haines T, et alA systematic review and meta-analysis of studies comparing mortality rates of private for-profit and private not-for-profit hospitals. CMAJ 2002;166: 1399-406. [Google Scholar]
49. North of England Evidence Based Guideline Development Project. Evidence based clinical practice guideline: the choice of antidepressants for depression in primary care. Newcastle upon Tyne: Centre for Health Services Research, 1997.