What is Evidence Synthesis?

Put simply, evidence synthesis “is a way of combining information from multiple studies that have investigated the same thing, to come to an overall understanding of what they found.” — Cochrane 

When asking Sam Cheng, a scientist at the Center for Biodiversity and Conservation who specializes in evidence syntheses approaches, she says:

“Evidence synthesis is often used to inform decisions – from policy and practice decisions in real life to research prioritization or developing new technologies, etc. There are many different methods to conduct evidence syntheses – all of which range in their overall rigor, reliability, and level of risk (as it pertains to how well it can reliably and comprehensively inform the decision at hand).

Thus, the term “evidence” can refer to many things. Generally in this case – we refer to “evidence” as knowledge from scientific studies, evaluations, observations, and experience. When this type of knowledge is synthesized in a comprehensive and systematic way that aims to be transparent and minimize bias, we often refer to these as systematic evidence syntheses. Systematic evidence syntheses include things like systematic reviews, systematic maps, and evidence gap maps. The value of using rigorous evidence synthesis methods is that it allows for clear assessment of the strength and reliability of different sources of information (because not all studies are created equal!) and avoids cherry-picking studies to inform decisions

• Systematic reviews are “a structured, step-wise methodology following an a priori protocol to comprehensively and systematically collate, critically appraise and synthesize existing research evidence (traditional academic and grey literature). This method is applicable to specific questions such as: What is the effectiveness of an intervention? What is the effect of X on Y? What is the prevalence of a phenomenon? How reliable is a specific method?” (Collins et al. 2015). Systematic reviews aim to be transparent in their methods which ideally are peer-reviewed and co-developed with a stakeholder group who will be end-users of the review (or representative of potential end-users) to ensure that they minimize any risk of bias and maximize comprehensiveness and useability. Specifically, a systematic review looks at the direction and magnitude of effects, and can also look into the influence of factors and contexts through critical appraisal methods (assessing the rigor and reliability of individual studies) and narrative and meta-analytic summaries of quantitative and qualitative data (see Meta-analyses section below).

• Systematic maps are similar to systematic reviews but they tend to look at broader topics to provide insight on where key research gaps may exist (identifying potential areas for future prioritization) as well as areas where there is a sufficient body of evidence for a systematic review or in-depth assessment (James et al. 2016). You can think of a systematic map as like a  bathymetry map that allows you to see where the peaks and troughs are along the ocean floor while a systematic review is like taking an ROV down one of the troughs and doing an in-depth, but a narrower assessment of what lives there and what it’s like.

• Meta-analyses are methods to summarize quantitative data from existing studies. This is often a method that is used within a systematic review and can also be done as a stand-alone. Meta-analyses, however, if they do not use systematic approaches for searching for and selecting studies for their analysis, can be prone to bias. So a good meta-analysis should follow the guidelines for a systematic review and utilize meta-analysis methods (with proper controls for heterogeneity and effect size) to summarize data across studies to provide insight on the question at hand.

If you think that your project could benefit from an evidence synthesis, you should consider learning more about the tools and approaches for doing these reviews. 

This website developed by the Stockholm Environment Institute has a variety of free-to-access training mini-courses in evidence synthesis. The courses cover synthesis methods, including systematic review and systematic mapping, and stakeholder engagement in evidence synthesis.

If you’d like to learn more about the meta-analysis approach, this collection of short videos introduces the ideas and steps of completing a meta-analysis.

Another tool is Colandr, which is an open access machine-learning assisted online platform for conducting reviews.

Other helpful resources include these articles:

Brief primer on scoping and preparing evidence synthesis proposals – by Sam Cheng

Collins, A., Miller, J., Coughlin, D., Kirk, S., (2014). The Production of Quick Scoping Reviews and Rapid Evidence Assessments: A How to Guide. Joint Water Evidence Group, UK.

Dicks, L., Haddaway, N., Hernández-Morcillo, M., Mattsson, B., Randall, N., Failler, P., … and Rodela, R. (2017). Knowledge synthesis for environmental decisions: an evaluation of existing methods, and guidance for their selection, use and development: a report from the EKLIPSE project.

James, K.L., Randall, N.P., Haddaway, N.R. (2016). A methodology for systematic mapping in environmental sciences. Environmental Evidence 5, 7.

Pullin, A.S., Cheng, S.H., Cooke, S.J., Haddaway, N.R., Macura, B., McKinnon, M.C., and Taylor, J.J. (2020). Informing conservation decisions through evidence synthesis and communication. Conservation Research, Policy and Practice, 114.