Study heterogeneity

Meta-analysis is a method used to combine the results of different trials in order to obtain a quantified synthesis. The size of individual clinical trials is often too small to detect treatment effects reliably. Meta-analysis increases the power of statistical analyses by pooling the results of all available trials.

Additional recommended knowledge

How to ensure accurate weighing results every day?

Essential Laboratory Skills Guide

Weighing the right way

As we are trying to use the meta-analysis to estimate a combined effect from a group of similar studies, we need to check that the effects found in the individual studies are similar enough that we are confident a combined estimate will be a meaningful description of the set of studies. In doing this, we need to remember that the individual estimates of treatment effect will vary by chance, because of randomization. Thus we expect some variation. What we need to know is whether there is more variation than we'd expect by chance alone. When this excessive variation occurs, we call it statistical heterogeneity, or just heterogeneity.

When there is heterogeneity that cannot readily be explained, one analytical approach is to incorporate it into a random effects model. A random effects meta-analysis model involves an assumption that the effects being estimated in the different studies are not identical, but follow some distribution. The model represents our lack of knowledge about why real, or apparent, treatment effects differ by considering the differences as if they were random. The centre of this symmetric distribution describes the average of the effects, while its width describes the degree of heterogeneity. The conventional choice of distribution is a normal distribution. It is difficult to establish the validity of any distributional assumption, and this is a common criticism of random effects meta-analyses. The importance of the particular assumed shape for this distribution is not known.

References

• Cochrane Handbook for Systematic Reviewers