Levels of evidence: can quality of systematic reviews be quantified?

non disponibile

Introduction One of the key theoretical principles of meta-analyses is that all data must be treated equally with precision. In recent years, however, the quality of the reporting of data in primary studies, often used as a proxy measure for methodological quality, has been shown to affect estimates of intervention efficacy reported in meta-analyses (Schulz et al., 1995; Moher et al., 1999; Tierney & Stewart, 2005; Gluud, 2006), although data are still controversial Emerson et al., 1990; Kjaergard et al., 2001; Balk et al., 2002; Juni et al., 2001). Meta-analysts need to take quality into consideration to reduce heterogeneity and to provide unbiased treatment estimates (Moher et al., 1999). In order to investigate whether different methods of quality assessment provide different estimates of intervention efficacy, Moher and colleagues randomly selected 11 meta-analyses (127 RCTs, mostly placebo-controlled) dealing with different medical areas (digestive diseases, circulatory diseases, mental health, neurology and pregnancy and childbirth) (Moher et al., 1999). A statistically significant exaggeration of treatment efficacy was found when results of lower-quality trials were pooled whether the trial quality assessments were made by a scale approach or by an individual component approach. However, generalisability of findings can be limited by whether or not there is an active comparator (heterogeneity of intervention, population and outcome) and furthermore 2 sensitivity analyses can miss to find possible confounding variables, apparently not related to trial quality. In the field of meta-analyses of data extracted from antidepressant (AD) RCTs, quality remains a hot issue. It is unclear whether in this specific field a relationship exists between quality measures and treatment estimates and, additionally, it is unclear whether different quality measures provide different estimates of treatment efficacy. Furthermore, to reliably inform clinical practice there is the need for grading the evidence coming from systematic reviews (and meta-analyses) in the field of AD treatment for major depression. To answer these questions, we therefore investigated the following issues in a step-wise approach: (1) whether RCT quality, assessed by either validated rating scales or individual components, influenced treatment estimates in a homogeneous sample of AD RCTs. An ongoing Cochrane review concerned with fluoxetine included published clinical trials comparing fluoxetine to other ADs, offered an opportunity for this analysis (Cipriani et al., 2006). (2) whether it is possible to find a validated way of grading the quality of systematic reviews (and meta-analyses) in order to have an explicit hierarchy of robustness and reliability of findings. An ongoing multiple treatment meta-analysis (MTM) was used to test this hypothesis.