
Systematic review guide | Categories, procedures, and obstacles
Systematic reviews are a structured way to analyze and synthesize research, offering reliable answers to specific questions. They follow strict methods to minimize bias and ensure accuracy, making them critical for evidence-based decisions in fields like healthcare and policy-making. Here's a quick breakdown:
- Definition: A systematic review uses rigorous methods to evaluate and combine evidence from multiple studies.
- Benefits: Provides clear answers, bridges research gaps, and supports clinical and policy decisions.
- Types: Includes meta-analysis (statistical synthesis), scoping reviews (broad mapping of topics), and mixed methods reviews (combining qualitative and quantitative data).
- Process: Starts with a clear research question, followed by protocol registration, literature search, data extraction, and analysis.
- Challenges: Includes managing diverse study designs, addressing bias, and dealing with time/resource limitations.
- Tools: Platforms like Sourcely and Covidence, along with frameworks like PRISMA, simplify the review process.
Systematic reviews are often considered the gold standard for synthesizing research, but they require careful planning, clear methodology, and the right tools to overcome common obstacles.
💪 Systematic Literature Review Made EASY: A Step-by-Step Guide
Types of Systematic Reviews
Systematic reviews come in various forms, each tailored to different research goals. Knowing these types helps you choose the right approach for your project and manage your expectations effectively.
Meta-Analysis
A meta-analysis focuses on combining quantitative data from multiple studies that address the same research question. This method allows for statistical conclusions to be drawn when the data across studies are comparable. While a systematic review identifies and evaluates relevant studies, a meta-analysis takes it further by adding a statistical layer to synthesize the findings.
This approach is particularly impactful in healthcare research. By pooling results from several clinical trials, meta-analyses can provide stronger evidence about the effectiveness of treatments compared to relying on individual studies alone.
Scoping Reviews
Scoping reviews serve a different purpose than systematic reviews. Instead of answering specific clinical questions, they aim to map the breadth of existing literature on a topic. These reviews are especially useful for identifying knowledge gaps, clarifying concepts, or exploring how research in a specific area is conducted.
For example:
- Schaink et al. explored definitions of "patient complexity".
- Callary et al. reviewed methodologies for assessing hip replacement wear.
Interestingly, only 12% of scoping reviews recommend conducting more focused systematic reviews afterward.
Qualitative and Mixed Methods Reviews
Qualitative reviews bring together themes and findings from qualitative studies, while mixed methods reviews combine both qualitative and quantitative data to provide a more comprehensive perspective on complex issues.
Conducting mixed methods reviews requires expertise to integrate these different types of data effectively. This includes careful planning and sufficient resources to manage the process. Key integration strategies include:
Integration Level | Key Approaches |
---|---|
Design | Exploratory sequential, Explanatory sequential, Convergent, Multistage |
Methods | Connecting, Building, Merging, Embedding |
Interpretation and Reporting | Narrative, Data transformation, Joint display |
However, these reviews come with challenges. The complexity of their design, the time and resources required, and the philosophical differences between qualitative and quantitative research paradigms can create significant hurdles. A review of mixed methods systematic reviews revealed that many authors struggle to adhere to established methodologies.
Ultimately, the type of review you choose should align with your research question and objectives. Carefully consider what you aim to uncover or achieve before deciding on your methodology.
With these review types explained, the next step is to dive into the process of conducting a systematic review.
Step-by-Step Systematic Review Process
Systematic reviews offer a structured way to synthesize research, but they require meticulous planning and execution. Below is a detailed guide to navigating each step of the process effectively.
Creating a Research Question
The foundation of any systematic review is a well-defined, answerable research question. Without clarity, the review risks becoming unfocused and unreliable. To craft a strong question, consider using a framework tailored to your study's needs. Different frameworks suit different types of research, so selecting the right one is key.
Framework | Best For | Key Elements |
---|---|---|
PICO | Quantitative reviews on interventions | Patient/Population, Intervention, Comparison, Outcome |
SPIDER | Qualitative and mixed methods research | Sample, Phenomenon of Interest, Design, Evaluation, Research type |
SPICE | Evaluating services or products (qualitative evidence) | Setting, Perspective, Intervention, Comparison, Evaluation |
ECLIPSE | Health policy and management | Expectation, Client group, Location, Impact, Professionals, Service |
PCC | Scoping reviews | Population, Concept, Context |
For example, PICO is widely used for quantitative studies, while frameworks like SPIDER or SPICE are better suited for qualitative research. Consider a review on UK healthcare workers' attitudes toward seasonal influenza vaccination. Researchers could define their population as healthcare workers across the UK, focus on the context of seasonal influenza vaccination, and gather qualitative data through interviews and focus groups. They might exclude studies on the general population or those addressing pandemic influenza vaccination, as well as purely quantitative studies.
Once your question is established, it serves as the foundation for developing and registering your review protocol.
Writing and Registering a Protocol
The protocol is your systematic review’s roadmap. It outlines the rationale, research questions, inclusion and exclusion criteria, search strategies, data extraction methods, quality assessment procedures, data synthesis techniques, and grading systems for evidence. Many journals now mandate protocol registration.
Registering your protocol on platforms like PROSPERO or OSF ensures transparency, minimizes bias, and prevents duplication of efforts. It’s a key step in maintaining the integrity of your review.
Literature Search and Screening
The strength of a systematic review lies in its literature search. Start by defining your search concepts based on your chosen framework. Use both multidisciplinary and subject-specific databases to ensure a thorough search. Develop a list of search terms, including synonyms, variations, and acronyms. Controlled vocabulary (like MeSH terms in PubMed) can improve precision, but don’t overlook keyword searches to capture variations not covered by controlled terms.
Combine search terms with Boolean operators (AND, OR) and use parentheses to structure the logic of your search. Documenting your search strategy ensures transparency and makes the process reproducible.
Screen titles and abstracts using clear inclusion and exclusion criteria. To enhance the comprehensiveness of your review, consider hand-searching reference lists of included studies and checking articles that have cited them. Including grey literature can also help reduce bias.
Once you’ve identified relevant studies, proceed to extract and evaluate the data systematically.
Data Extraction and Quality Assessment
Standardized forms and multiple independent reviewers are crucial for extracting and assessing data. Any discrepancies should be resolved through discussion or with the help of a third-party reviewer to maintain objectivity.
Data Analysis and Reporting Results
Your approach to data analysis will depend on your research question and the types of studies included. Be mindful of publication bias when interpreting results.
For quantitative data, a statistical meta-analysis may be appropriate if the studies are sufficiently similar. When dealing with qualitative data or diverse quantitative studies, narrative synthesis provides a structured way to summarize findings. Following established reporting standards, such as PRISMA, ensures your review meets global benchmarks for transparency and completeness. This allows readers to evaluate the reliability of your conclusions.
sbb-itb-f7d34da
Common Problems in Systematic Reviews
Even with careful planning, systematic reviews often face challenges that can affect their quality and reliability. Recognizing these issues and addressing them effectively is essential for producing trustworthy research.
Different Study Types and Methods
Bringing together studies with different designs in one review can be tricky. When your research question requires evidence from various types of studies, you'll need to handle differences in how data is collected, outcomes are measured, and analyses are conducted.
Mixed methods systematic reviews can provide a broader understanding of complex topics but come with added layers of difficulty. For instance, when combining quantitative and qualitative data, a convergent integrated approach (transforming data) works well. If you're examining distinct aspects of a topic, a convergent segregated approach with separate syntheses is a better fit.
However, these approaches often involve challenges such as reviewing a larger number of citations, longer search times, inconsistent methodologies, and higher resource demands. To navigate these complexities, tailor your study designs to answer your specific questions. Critical appraisal tools, like the Mixed Methods Appraisal Tool (MMAT), can help evaluate diverse study types effectively.
Bias and Selective Reporting
Bias poses a significant threat to the validity of systematic reviews. Unfortunately, many reviews fail to properly assess or even address bias.
"Bias in research refers to a systematic error that can occur during the design, conduct, or interpretation of a study, leading to inaccurate conclusions." - Critical Appraisal Skills Programme (CASP)
The numbers are alarming. A review of environmental articles published between 2018 and 2020 found that 64% did not assess risk of bias. In healthcare AI research, 50% of studies had a high risk of bias, and for neuroimaging-based AI models in psychiatric diagnosis, 83% were rated as high risk.
Bias Type | Description | Mitigation Strategies |
---|---|---|
Selection Bias | Systematic differences between groups' baseline characteristics | Use random sequence generation and allocation concealment |
Attrition Bias | Bias caused by participants dropping out of the study | Address withdrawal imbalances between study groups |
Detection Bias | Differences in outcome measurement across groups | Mask outcome assessors for consistent evaluations |
Reporting Bias | Selective reporting of significant results while ignoring others | Follow a pre-specified protocol for outcomes |
Publication Bias | Research results not fully represented in published literature | Use tools like Egger's test and funnel plots |
To reduce bias from missing evidence, include results from unpublished sources. Pay special attention to studies with missing data that could influence outcomes based on p values, effect sizes, or directions. Ensuring a comprehensive search across all potential sources is also critical.
Time and Resource Limits
Beyond methodological and bias-related challenges, resource constraints are a major hurdle. Systematic reviews are notoriously time-intensive. On average, they require 881 person-hours, with durations ranging from 243 to 1,752 hours depending on the scope. Most reviews take up to 2 years to complete, with project management and administrative tasks taking up a significant portion of the time.
Efficiency can vary greatly. For example, one review was completed in just 61 person-hours using automation tools. The time required often depends on the methods and resources available.
Here’s how time and costs can differ based on specific strategies:
- Semiautomated screening: Requires 572 hours and costs around $51,500.
- Dual screening: Takes about 1,089 hours and costs nearly $102,200.
- Single data abstraction with verification: Averages 107 minutes per study.
- Dual independent data abstraction: Takes about 172 minutes per study.
To save time, consider using two monitors for data extraction, hiring experienced information specialists for literature searches, and leveraging collaborative software. Blocking dedicated time for the review and holding daily check-ins can keep the project on track. Writing your protocol in past tense can also streamline the process.
Lack of experience, insufficient domain knowledge, and poor communication can quickly increase time and resource demands. Investing in skilled team members and the right tools often results in better efficiency and higher-quality outcomes.
Tools and Resources for Systematic Reviews
Using the right tools can make systematic reviews faster, easier, and more accurate. These resources not only streamline the process but also help tackle common challenges. Many of these tools align perfectly with the systematic review process discussed earlier.
Sourcely: AI-Powered Academic Tool
Sourcely is a standout AI-powered platform designed to simplify the task of finding and managing academic sources. With access to over 200 million papers, it’s trusted by top institutions like Harvard, MIT, and Stanford [45,46]. The platform excels at matching sources to your research. For instance, you can paste a research question or a draft, and Sourcely will quickly identify relevant sources. In a test conducted by researcher Sophia Sipos, uploading a 1,000-word paper generated a detailed source list in just 10 seconds, and key citation text was highlighted in 20 seconds.
Some of its key features include:
- Advanced search filters: Narrow results by publication year, authorship, or relevance.
- Citation library: Easily organize sources and get quick overviews.
- Source access: Download free PDFs for many papers and export references in multiple formats for bibliographies.
Sourcely has earned high praise from users, with a 4.8/5 rating on Capterra (based on 300+ reviews) and a 4.5/5 rating on GetApp (from over 50 reviews). It’s available starting at $7 for a trial, with monthly and multi-year plans also offered.
Guidelines and Frameworks
Frameworks are essential for conducting systematic reviews with rigor and consistency. The PRISMA 2020 statement is a widely used guide for reporting the rationale, methods, and findings of systematic reviews [49,50]. It replaces the earlier PRISMA 2009 statement, which had been cited in over 60,000 reports as of August 2020. For more detailed methodology and best practices, the Cochrane Handbook is another key resource.
In addition to these frameworks, specialized software can simplify citation management and data handling, ensuring a smoother review process.
Reference Management and Analysis Software
Managing references and data is much easier with specialized tools. Covidence, for example, is specifically built for review management and can reduce the time spent on reviews by an average of 35%, saving about 71 hours per review.
Other tools like EndNote, Mendeley, and Zotero are excellent for citation management. They help remove duplicates, allow manual or batch addition of references, and enable collaborative work by sharing library access. These platforms also let you tag, edit entries, and store full-text articles. Additionally, they track metrics like the number of references identified, duplicates removed, and full-text articles reviewed.
For more advanced needs, software like EPPI-Reviewer, DistillerSR, SUMARI, Rayyan QCRI, and SysRev offers tailored features to support systematic reviews. It’s a good idea to familiarize yourself with these tools in advance to ensure smooth integration into your workflow.
Conclusion and Key Points
Conducting systematic reviews effectively requires a well-structured and disciplined approach. These reviews are often considered the gold standard for synthesizing evidence. However, their reliability hinges on adhering to established methodologies and tackling common challenges head-on.
The process begins with defining a clear research question and registering a protocol early in the project. Skipping this foundational step can lead to issues like scope creep or inconsistencies in methodology, which can undermine the review's credibility.
Bias is another significant hurdle. To minimize it, reviewers should make precise methodological choices and seek independent oversight. Expanding the scope of evidence by including trial registries and regulatory documents - beyond just published studies - can also help address the problem of missing or incomplete data.
Time and resource constraints are challenges nearly every review team faces. On average, it takes 67 weeks from protocol registration to publication for registered reviews. To streamline this lengthy process, involving an experienced information specialist early on and using tools for reference management and data extraction can make a significant difference. Careful planning and resource optimization are key to maintaining quality throughout the review process.
Practical challenges also demand attention. Thorough preparation and expertise play a critical role in overcoming obstacles. Double-checking data, consulting statistical experts for complex analyses, and fostering clear communication within the team can prevent costly errors and ensure a smoother workflow.
Ultimately, success in systematic reviews boils down to balancing rigorous methodology with practical execution. While guidelines provide a roadmap, the real challenge lies in applying these principles effectively while acknowledging any limitations. Comprehensive reviews are indispensable for high-stakes decisions, like national guidelines or drug licensing. However, in situations demanding quicker results, rapid review methods can provide timely, actionable insights without sacrificing quality.
FAQs
What is the difference between a systematic review and a scoping review, and how do you decide which one to use?
A systematic review focuses on answering a specific, clearly defined research question by thoroughly evaluating and synthesizing evidence from selected studies. This approach is perfect when you need precise, reliable conclusions based on a detailed and rigorous analysis.
In contrast, a scoping review takes a broader approach, offering an overview of a large or diverse body of literature. It’s particularly useful for identifying key concepts, spotting research gaps, and understanding the range of available evidence. This makes it an excellent tool for examining emerging topics or addressing broader research questions.
Typically, scoping reviews are conducted first to map out the existing research landscape. Systematic reviews then follow, diving deeper into evidence synthesis once a well-defined research question is in place.
What are the best strategies to reduce bias and maintain accuracy in a systematic review?
To ensure a systematic review remains accurate and fair, researchers need to establish clear inclusion and exclusion criteria. This helps maintain consistency when deciding which studies to include. Additionally, assessing the risk of bias in each study using standardized tools is a critical step in maintaining objectivity.
Other helpful practices include using blinding during the review process, sticking to well-defined protocols, and offering comprehensive training for the research team. These measures work together to boost the reliability of the review and contribute to producing high-quality findings.
What are effective ways to manage time and resources when conducting a systematic review?
Managing time and resources during a systematic review can feel overwhelming, but there are ways to make the process smoother. One option is to use specialized software or AI tools to handle repetitive tasks like screening studies or extracting data - this can save a lot of time. Rapid review methods are another way to speed things up by concentrating on the most essential parts of the review.
It's also helpful to start with a well-organized plan. Set clear goals, break the work into manageable tasks, and prioritize what needs to be done first. For more involved reviews, working with a team or consulting experts can help you stay on track and maintain quality without overloading your resources.