How to do an Evidence Synthesis


What is Evidence Synthesis?

Put simply, evidence synthesis “is a way of combining information from multiple studies that have investigated the same thing, to come to an overall understanding of what they found.” — Cochrane 

When asking Sam Cheng, a scientist at the Center for Biodiversity and Conservation who specializes in evidence syntheses approaches, she says:

“Evidence synthesis is often used to inform decisions – from policy and practice decisions in real life to research prioritization or developing new technologies, etc. There are many different methods to conduct evidence syntheses – all of which range in their overall rigor, reliability, and level of risk (as it pertains to how well it can reliably and comprehensively inform the decision at hand).

  • Systematic reviews are “a structured, step-wise methodology following an a priori protocol to comprehensively and systematically collate, critically appraise and synthesize existing research evidence (traditional academic and grey literature). This method is applicable to specific questions such as: What is the effectiveness of an intervention? What is the effect of X on Y? What is the prevalence of a phenomenon? How reliable is a specific method?” (Collins et al. 2015). Systematic reviews aim to be transparent in their methods which ideally are peer-reviewed and co-developed with a stakeholder group who will be end-users of the review (or representative of potential end-users) to ensure that they minimize any risk of bias and maximize comprehensiveness and useability. Specifically, a systematic review looks at the direction and magnitude of effects, and can also look into the influence of factors and contexts through critical appraisal methods (assessing the rigor and reliability of individual studies) and narrative and meta-analytic summaries of quantitative and qualitative data (see Meta-analyses section below).
  • Systematic maps are similar to systematic reviews but they tend to look at broader topics to provide insight on where key research gaps may exist (identifying potential areas for future prioritization) as well as areas where there is a sufficient body of evidence for a systematic review or in-depth assessment (James et al. 2016). You can think of a systematic map as like a  bathymetry map that allows you to see where the peaks and troughs are along the ocean floor while a systematic review is like taking an ROV down one of the troughs and doing an in-depth, but a narrower assessment of what lives there and what it’s like.
  • Meta-analyses are methods to summarize quantitative data from existing studies. This is often a method that is used within a systematic review and can also be done as a stand-alone. Meta-analyses, however, if they do not use systematic approaches for searching for and selecting studies for their analysis, can be prone to bias. So a good meta-analysis should follow the guidelines for a systematic review and utilize meta-analysis methods (with proper controls for heterogeneity and effect size) to summarize data across studies to provide insight on the question at hand.

If you think that your project could benefit from an evidence synthesis, you should consider learning more about the tools and approaches for doing these reviews. 

If you’d like to learn more about the meta-analysis approach, this collection of short videos introduces the ideas and steps of completing a meta-analysis.

  • Another tool is Colandr, which is an open access machine-learning assisted online platform for conducting reviews.

Other helpful resources include these articles: