Synthesis

Phase 4: Synthesise the evidence

Synthesis turns an extraction spreadsheet into an answer. The right method depends on the data you have, the question you asked, and the review type you chose. Quantitative, homogeneous outcomes can be pooled in a meta-analysis; heterogeneous quantitative outcomes call for narrative synthesis; qualitative studies require thematic or meta-synthetic methods. This phase is where methodological training matters most, and where many reviews fall back on "vote counting," which is not synthesis at all.

1. Narrative synthesis (Popay et al. framework)

The Popay et al. (2006) Guidance on the Conduct of Narrative Synthesis is the de facto standard for non-meta-analytic synthesis in systematic reviews. It has four elements:

1. Developing a theory of how, why, and for whom the intervention works.
2. Developing a preliminary synthesis using textual descriptions, groupings, tabulation, and transformations (e.g., converting different effect measures to a common direction of effect).
3. Exploring relationships within and between studies, including moderator variables, conceptual models, and reciprocal translation.
4. Assessing the robustness of the synthesis (methodological quality, reviewer reflexivity, sensitivity to choices).

Narrative synthesis is the default for scoping reviews and rapid reviews, and for systematic reviews where meta-analysis is not appropriate.

2. Thematic synthesis (Thomas & Harden)

Thomas & Harden's (2008) thematic synthesis is the canonical method for qualitative systematic reviews of intervention effectiveness and acceptability. It has three steps:

1. Line-by-line coding of the findings of primary qualitative studies.
2. Development of descriptive themes that remain close to the original findings.
3. Generation of analytical themes that go beyond the primary studies to answer the review question.

Thematic synthesis is auditable, replicable, and well suited to reviews that combine quantitative effectiveness evidence with qualitative evidence on acceptability, feasibility, or experience.

3. Framework synthesis

Framework synthesis (Dixon-Woods, 2011) is deductive: an a priori framework (a theory, a policy model, a logic model) structures the analysis. Studies are coded against the framework; gaps and modifications emerge. Framework synthesis is fast, transparent, and well suited to policy-relevant scoping and rapid reviews where time is constrained.

4. Meta-analysis

When multiple studies report a comparable outcome, effect sizes can be statistically pooled.

• Effect sizes — standardised mean difference (SMD) for continuous outcomes, risk ratio (RR) or odds ratio (OR) for binary outcomes, hazard ratio (HR) for time-to-event.
• Fixed-effect models assume a single true effect across studies; appropriate when studies are clinically and methodologically homogeneous.
• Random-effects models assume a distribution of true effects; appropriate when heterogeneity is expected. DerSimonian-Laird is the classic estimator; REML is preferred in modern software.
• Heterogeneity — assessed with Cochran's Q (statistical test) and I² (proportion of total variability due to heterogeneity rather than chance). Rough I² benchmarks: 0–40% might not be important, 30–60% moderate, 50–90% substantial, 75–100% considerable.
• Forest plots visualise each study's effect and 95% CI, the pooled estimate (diamond), and heterogeneity statistics.
• Publication bias — funnel plots and Egger's test are standard; consider trim-and-fill or selection models when ≥ 10 studies are pooled.
• Subgroup and meta-regression — explore sources of heterogeneity, pre-registered in the protocol.

For a deeper statistical treatment of pooled estimates, effect size conversions, and model choice, see Stats for Scholars as a companion resource. Meta-analysis is the defining method of meta-analytic reviews and a frequent component of systematic reviews.

5. Qualitative meta-synthesis

Qualitative meta-synthesis goes beyond aggregation to produce new interpretive theory. Common approaches:

• Meta-ethnography (Noblit & Hare, 1988) — reciprocal translation and lines-of-argument synthesis.
• Critical interpretive synthesis — reflexive, theory-generating, suited to complex policy questions.
• Meta-aggregation (JBI) — pragmatic pooling of findings into categories and synthesised findings.

For methodological detail on qualitative synthesis, sampling, and reflexivity, see The Qualitative Researcher as a companion resource.

6. Assessing certainty with GRADE

GRADE (Grading of Recommendations Assessment, Development and Evaluation) rates the certainty of the body of evidence for each outcome, not individual studies. The starting certainty is high for RCTs and low for observational studies, then downgraded or upgraded across eight domains:

Final ratings are high, moderate, low, or very low. A Summary of Findings (SoF) table presents GRADE ratings alongside absolute and relative effects for each outcome. GRADE-CERQual is the parallel framework for qualitative evidence syntheses.

7. Presenting the synthesis

Whichever method you use, the results section should present:

• A flow of included studies (summarised from your PRISMA flow diagram).
• Characteristics-of-included-studies table.
• Risk-of-bias figure (traffic light or summary).
• The synthesis itself — forest plot, narrative theme map, or framework matrix.
• Certainty of evidence (GRADE / CERQual).
• Sensitivity, subgroup, and publication-bias analyses.

8. Common synthesis pitfalls

• Vote counting (counting positive vs negative studies without weighting by effect size, sample size, or quality). Not synthesis.
• Pooling heterogeneous outcomes into a single meta-analysis because the software allowed it.
• Ignoring I² and reporting only the pooled point estimate.
• Mixing qualitative paradigms without explicit justification.
• Skipping GRADE — reviewers and guideline developers expect it.

Tools and templates for this phase

• Literature Review Matrix template — for narrative and framework synthesis.
• Related review types: systematic reviews, meta-analyses, integrative reviews, umbrella reviews.
• Phase links: extraction produces your synthesis input; reporting packages your synthesis output.

Next phase

With your synthesis complete, move to Phase 5 — Reporting →