More than just evidence summaries — rating the quality of evidence with GRADE

NPS MedicineWise is proud to join other evidence-based organisations, including the WHO and Cochrane Collaboration, in adopting the Grades of recommendation, assessment, development, and evaluation (GRADE) framework for our various publications, starting with our 'flagship' NPS RADAR.

In the June 2014 RADAR review on tapentadol we use the GRADE criteria to discuss the quality of evidence used in developing our conclusions about benefits and harms.

The GRADE approach has been endorsed by many national and international organisations as a systematic and transparent framework for communicating the quality of evidence and strength of recommendations in healthcare.

GRADE system for classifying quality of evidence

To achieve transparency and simplicity the GRADE system classifies evidence in one of four levels — high, moderate, low, and very low.¹

High quality — Further research is very unlikely to change our confidence in the estimate of effect.

Moderate quality — Further research is likely to have an important impact on our confidence in the estimate of effect and may change the estimate.

Low quality — Further research is very likely to have an important impact on our confidence in the estimate of effect and is likely to change the estimate.

Very low quality — Any estimate of effect is uncertain. ¹

Going beyond levels of evidence to objectively assess its quality

In line with the GRADE framework, NPS has always considered randomised controlled trials as high-quality evidence, and observational studies typically would only be presented when data from RCTs are limited or not available.

In our critical appraisal we systematically evaluate clinical studies to establish that the question satisfies the four elements of PICO — population, intervention, comparison, and outcome — and that the methodology, analysis and interpretation of the effects are valid.

The GRADE framework takes this a step further by providing a systematic approach to assess confidence in the evidence, based on five objective criteria known as the GRADE criteria:¹

• risk of bias — are there limitations in the detailed study design and execution?
• inconsistency — how consistent or homogeneous are the estimated effects between trials?
• indirectness — is the PICO appropriate for the clinical claim? How applicable are the data to the Australian context?
• imprecision —how wide are the confidence intervals around the estimate of the effects?
• publication bias — is there any evidence that results or data have been selectively published?

When RCTs are critically appraised for RADAR reviews we will now provide a summary and conclusion on the quality of the evidence by considering these GRADE criteria and quality levels.