So you have an idea for a systematic literature review that you believe will:
- answer an important research question
- advance the knowledge in your field
- help physicians when selecting appropriate treatment regimens
- give policy-makers the information they need to promote public health in a meaningful way
- get you a cool publication that will be cited extensively for a long time.
Performing and publishing this amazing systematic review may get you a new job or finally convince your department to let you graduate. Terrific! But what do you do now?
What to do BEFORE you put the P.I.E.C.E.S. together:
- Determine the feasibility of your idea. Look for original studies in that area of research. Are there enough studies for a systematic review?
- Find and evaluate related systematic reviews. Has this question been answered already? When was it answered? How much original research has been published since? Is it time for an updated systematic review? Should you use different criteria for selecting the studies?
- Assemble your team and decide the responsibilities of each person. Who will select the studies for inclusion in the review? Who will adjudicate the differences between the people who select the studies? Who will produce the graphics? Who will do the statistical analysis? Who will write the manuscript and be the first author?
- Decide the general research question and which guidelines you will follow (e.g. PRISMA).
P.I.E.C.E.S. = Plan, Identify, Evaluate, Collect, Explain, Summarize
(Foster & Jewell, 2017)
P = Plan: decide on your search methods
- Develop a protocol. Formulate a precise research question, hypothesis, eligibility criteria, and scope.
- Decide which databases you will search and which search terms to use. You can perform pilot searches to test keywords, search strings, and code.
- Define the eligibility criteria for studies. These are the reasons to include and to exclude studies. Decide the range of publication dates and the languages of the included studies.
- Determine what outcomes and variables you will record, and create a form or table for logging the relevant information from each eligible study.
- Select the methods you will use to interpret the data and evaluate the quality of each study and the risk of bias.
- Optional: pre-register the study in an online registry such as PROSPERO.
- Optional: write a protocol manuscript and submit it to a peer-reviewed journal.
I = Identify: search for studies that match your criteria
- Run the searches according to the search terms and combine the results from the source databases into one spreadsheet.
- Check the references lists of previous reviews to ensure that you do not overlook potentially relevant studies.
- Remove duplicates. Note how many publications the search has returned.
E = Evaluate: exclude or include studies
- Screen the publications by title and abstract. Exclude the publications that do not meet the scope and criteria of the systematic review. Note how many publications you excluded at this stage. This step can be performed by one person.
- Obtain the full-text documents of the remaining studies.
- Read the full text of each publication. Exclude the publications that do not meet the scope and criteria of the review. Note how many publications you excluded at this stage. This step must be performed by two people, each one working alone.
- Compare your list of included studies with the list made by the second person. Are your lists identical? If not, one of you excluded a study that the other person considers relevant. Discuss the discrepancy. If you cannot reach an agreement regarding the relevance of the study, ask a third person to read the study and make a decision. This person is typically the principal investigator or project manager in your group.
- How many included studies remain? Graph the exclusion process in a flow diagram.
C = Collect: extract and synthesize key data
- Use your form or table to log the relevant information from each study.
- Describe, summarize, categorize, and analyze the body of data. Sometimes researchers will use one study for two publications; exclude one of the articles and ensure that you are not counting the same patient population twice.
- If you have enough high-quality numerical data, you can conduct a quantitative synthesis (meta-analysis) to strengthen the quality of evidence in answering the research question by pooling the data.
- Identify studies with questionable data. If key data are incomplete or missing, contact the authors of the study and request the information. If you do not receive a reply, exclude the study from your analysis. At this stage, you can also exclude studies if the randomization was performed incorrectly, if you identify an error in the study design or methods, or if you see an ethical breach or notice bias related to conflicts of interest. Update your flowchart to reflect these exclusions.
- You can perform a sensitivity analysis with and without the data from the questionable publications, to see whether these data change the overall findings of your meta-analysis.
- Using the information from your final set of included studies, draw conclusions from your findings.
E = Explain: give context and rate the strength of the studies
- Assess your review for quality and risk of bias.
- Prepare display items to accompany your review, such as a funnel plot for heterogeneity, a forest plot for each outcome of interest, summary tables, and the study/patient flowchart.
- Put the overall findings into the context of the scientific or medical field and look at the strengths and limitations of the studies.
S = Summarize: write and publish your final report
- Complete the requirements of the structured guidelines you selected before beginning the review process.
- Write the manuscript in the structure of an original research study (introduction, methods, results, and discussion). Describe your methods and results clearly and transparently.
- If you published a review protocol, the same journal may accept the full review manuscript.
- Submit the manuscript to a suitable peer-reviewed journal.
What to do AFTER you put the P.I.E.C.E.S. together
- If you pre-registered the review in an online registry, update the registry record to reflect that the study was completed and published. Provide the reference and URL of the published review.
- Depending on journal policy, you may be able to share your datasets in a public repository.
- Promote your published systematic review to the scientific and medical communities. You’ve worked hard to advance the knowledge base—ensure that all interested parties know that your review is available!
Dr Dean Meyer has a background in environmental science with a specialist interest in toxicology and public health. Her doctoral research work focused on molecular mechanisms of metal detoxification in an invertebrate model. Her other research interests include the mechanisms of toxicity and disease causation, and the occupational sources of xenobiotics and their physiological effects.
Dr Meyer spent 8 years working at the Centers for Disease Control and Prevention in Atlanta, and has an extensive background in the areas of laboratory safety and environmental health.
She is a certified Editor in the Life Sciences (ELS) and joined Edanz Group as an editor in 2015.