Performing data extraction and analysis
Retrieve and select studies
At least two reviewers are needed to retrieve the studies according to the inclusion and exclusion criteria, and to discuss borderline cases. The kappa statistic may be reported to show inter-reviewer agreement. In some cases, the titles and abstracts may be enough to exclude a study, but in other cases the full paper must be retrieved and read. Unclear details may require contacting the original authors for clarification.
Extract relevant data
A summary table is made of the main features of the eligible studies, including methodology, study/sample characteristics, interventions, treatment arms, main outcome measures, and main results from data analyses. This table allows easy grouping and comparison of variables. Systematic reviews commonly provide a similar table in their descriptive section.
Evaluate study quality
Included studies are assessed for study methodology, changes in the protocol, biases, flaws, and lack of proper randomization (in randomized controlled trials). Low-quality studies and studies with duplicated data are excluded. If some data appear to be incomplete but usable, the original authors of the research article should be contacted to supply missing or raw data or to clarify information such as patient recruitment and randomization protocols. Study sponsors/funders could be approached to clarify issues and to supply relevant unpublished data.
Analyze the data
Data from the included studies are analyzed. In qualitative analyses, the common features, biases, and relationships within and between studies are identified and presented in a narrative format. The interpretation of findings should address limitations, competing interpretations, evaluation of the evidence, recommendations for practice, and directions for future research.
In quantitative analyses (meta-analyses), data are pooled and statistically tested with predefined tests. Results are standardized and summarized in terms of both absolute and relative effects. Effect size (eg, odds ratio, risk ratio, hazard ratio, or risk difference, with 95% confidence intervals) is also calculated.
Generate forest plots and funnel plots
Results are typically presented as a forest plot, which shows summary data from each included study and overall effect size and direction.
Additional analyses are performed to test for study variability (heterogeneity) and to weight the findings if needed. Sub-analyses may be needed if variables differ between studies (eg, by dose, age, or timing). The robustness of the analysis may be tested by both “fixed-effects” and “random-effects” models.
Forest plot example 1
https://commons.wikimedia.org/wiki/File:Generic_forest_plot.png
Forest plot example 2
(see Figure 2 in the paper): https://rbej.biomedcentral.com/articles/10.1186/s12958-019-0460-4
Publication bias and heterogeneity can be visualized in a funnel plot. Biasing effects of possibly skewed studies or very large studies can be tested by a sensitivity analysis (ie, by performing the analysis before and after omitting a specific study).
Funnel plot example 1
https://en.wikipedia.org/wiki/File:Funnelplot.png (Public Domain).
Funnel plot example 2
https://ijponline.biomedcentral.com/articles/10.1186/s13052-018-0586-6
WANT TO LEARN MORE?
To learn more about data extraction and analysis try our fourth course, SLR4: Performing Systematic Reviews
