As for every work of research, the research question is the crucial step that guides the choice of the paradigm and of the methodology of research. The research question can come from a dilemma inherent to the clinical practice, a gap in the practical or theorethical knowledge, or a mix of these elements.[3] The systematic review is an approach that can be included in quantitative methodologies, because it shares the main features of this methodology, such as hypothesis testing, numerical data collection and analysis and the role of the researcher detached from the research.
Search strategy and location of studies
The search strategy is the way in which the researcher runs the elctronic search to select the articles to include in the review. There is not a list of search strategies among which we can choose one. In fact, a systematic review is supposed to describe in details the search strategy that was used, as it influences the quality of the study. The most important elements of a search strategy are:
– balancing sensitivity and specifity. Where sensitivity means finding a high proportion of relevant studies, and specificity means finding a low proportion of irrelevant studies. [1]
– selecting databases (the presence of a thesaurus improves the coverage) [4]
– identifying MeSH terms to search, related to the research question or research framework (for example PICO).
– using database-appropriate syntax (for example parentheses, Boolean operators, field codes, variations in search terms)
Moreover, a search can also include other actions rather than electronic database search, such as reference lists checking and direct searching on key journals.
Inclusion and exclusion criteria
Inclusion and exclusion criteria interest the contents of the studies selected and their design. For the contents the PICO (Population, Intervention, Comparison, Outcomes) framework can be used. It summarizes the main components to consider when selecting studies. On the base of the objectives of the research, the researcher puts some inclusion and exclusion criteria for each of these components. The PICO framework is a useful tool for the inclusion/exclusion startegy, but it is not mandatory, as there are other strategies that are effective (for example the SPIDER framework for qualitative evidence synthesis). It is up to the researcher to adopt the most appropriate strategy for the research question. About the study design, the researcher decides a priori what types of studies to include and exclude, for example RCT, quasi experimental trials, cohort studies or combinations of them. More criteria can be added, for example language restrictions, free-acces studies, or published versus unpublished studies.[1]
After having obtained a list of titles and abstracts, the studies that meet inclusion criteria are then reviewed in full. To ensure inter-rater reliability the review process has to be done by two or more researchers. All reviewed studies should be recorded with thier own reasons dor inclusion and exclusion. [1]
Study selection
Study selection starts with the review of the abstracts that have been retrieved by the search strategy. The full-text of the studies that seem to meet inclusion criteria are obtained and reviewed. This process is preferrably done by at least two reviewers to establish inter-rater reliability.
A useful strategy is for authors to keep a log of all reviewed studies with reasons for inclusion or exclusion. In case more information about the study is required study authors may be contacted for information needed for data pooling (e.g., means, standard deviations). Translations may also be required.
Data extraction
The data extraction follows the identification of eligible studies. The aim of this step is to gain information about the included studies, and, in the case of a metanalysis, to obtain the quantitative data to carry out the statistical analysis. The information to be extracted from the studies are choosen by the researcher. The goal is to provide all the information necessary to generalise the results.[5]
Quality assessment
A systematic reviews evaluates the quality of the studies included through a risk of bias assessment. Different study designs have different risk of bias assessment tools. Here some examples of tools that can be used for three study designs:
– RCT: Cochrane Risk of Bias tool (ROB)
– case control studies: JBI Checklist for Case-Control Studies
– cohort studies: CASP- Cohort Studies
Data analysis and synthesis
For meta-analyses, statistical programmes are commonly used to calculate effects sizes.
An example of a statistical programme for data review analyses is the Review Manager (RevMan) endorsed by the Cochrane Collaboration.
Reporting of effect sizes is combined with a 95 % confidence interval (CI) range, and presented in both quantitative format and graphical representation (e.g., forest plots).
Forest plots visually depict each trial as a horizontal diamond shape with the middle representing the effect size (e.g., SMD) and the end points representing both ends of the CI. These diamonds are presented on a graph with a centre line representing the zero mark. Often the left side of the graph (< zero) represents the side favoring treatment, while the right side (> zero) represents the side favouring the control condition. At the bottom of the graph is a summary effect size or diamond representing all of the individual studies pooled together. Ideally, we would like to see this entire diamond (effect size and both anchors of the CI) falling below zero, indicating that the intervention is favoured over the control. In addition, most programs also calculate a heterogeneity value to indicate whether the individual studies are similar enough to compare. In this case, it is preferable to have non-significant findings for heterogeneity. It is still possible to pool studies when significant heterogeneity exists, although these results should be interpreted with caution or reasons for the heterogeneity should be explored.
Interpretation of the synthesis of study results is the last step in the writing process. This process usually involves summarising the findings, and providing clinical and future research recommendations.