Rapid Realist Review

What's its purpose in systems grantmaking?

To identify which interventions are most likely to influence the system in a specific context, what mechanisms need to be put in place first, and a few simple rules to guide systems change.

What is it?

A narrative reporting resource used in 10 rapid steps to quickly understand what interventions are likely to change a system, under what circumstances, through which mechanisms (i.e., structures, processes, activities) and with whom. The 10 steps, of which steps three through seven are iterative, include the following:

Recruit reference and expert panels and develop the project scope.
Develop the specific research questions.
Identify how the findings and recommendations will be used.
Develop search terms.
Identify articles and documents for inclusion in the review.
Conduct quality scoping literature review.
Extract data from the literature and appraise and synthesize the evidence.
Validate findings with content experts.
Synthesize the findings into a final report.
Disseminate the results.

When is it useful?

When there is a specific systems intervention or question.
In rapidly changing, emergent environments where a large amount of information needs to be processed and decisions made quickly.
It has been used most in health care and for policymaking but could be applied to other issues.

Example from the field

How does Rapid Realist Review work?

As part of an effort to transform the Saskatchewan provincial health system, the provincial Ministry of Health, funded by the Canadian Institutes of Health Research, worked with a research team1 to conduct a Rapid Realist Review about the role of government in conducting large-scale systems change in health care. The RRR identified five simple rules to guide Saskatchewan in its health system transformation efforts. These efforts, which included a number of initiatives, resulted in several improved outcomes such as decreased wait times for elective surgery and improved primary care.

The ministry went through RRR’s 10-step process, as described below, in six months.

Step 1: Ministry staff members began by recruiting for a reference (knowledge users) panel and an expert panel. They then worked with the knowledge users (i.e., those who implement or are impacted by the policy) and content experts to create a scope for the project. They defined the scope as understanding models and strategies, principles of partnership, and evaluation for large-scale systems change as well as identifying recommendations for the government’s role related to four specific initiatives (e.g., surgical wait lists).

Step 2: They cocreated four major research questions focused on the social processes that drive systems change, the contexts that impact this change, the transition points in change, and the role of government.

Step 3: They identified how the findings would inform the design of health care reform.

Steps 4 to 5: They selected relevant search terms for the review, and used them to identify articles and literature. They also asked experts and knowledge users for resources. They revised the search terms and identified additional literature. The team ultimately gathered more than 410 potential resources.

Step 6: They selected a sampling of 84 documents to review.

Step 7: They reviewed the documents using a template for data extraction to identify interventions, contexts, mechanisms and outcomes.

Step 8: They surveyed 44 field experts to validate the findings and fill gaps in the literature.

Steps 9 to 10: They synthesized and disseminated the findings in a report, which included actual recommendations for government involvement in the system change.

1. A list of all of the team members can be found here: Allan Best et al., “Knowledge and Action for System Transformation (KAST): A Systematic Realist Review and Evidence Synthesis of the Role of Government Policy in Coordinating Large System Transformation,” Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute. (2010).

What are tips and cautions for systems grantmakers and the social sector?

This is an alternative to traditional policy research and analysis or landscape scans. It explores what works for whom under what circumstances.
Since the information gathered is historical and current, newly emerging and untested future ideas are not captured. Thus, RRR recommendations may need to be complemented with other methods to generate innovations.
Some fields may not contain enough published or grey literature to adequately draw conclusions about relationships among context, mechanisms and outcomes.
While not participatory, RRR incorporates contributions from content experts and knowledge users. Those contributions are key to ensuring that RRR captures emerging ideas not yet in the literature, and validates the findings. If it isn’t possible for grantmakers to survey these individuals, the data may be limited and biased.

Resources

A Time-Responsive Tool for Informing Policy Making: Rapid Realist Review

By Jessie E. Saul, Cameron D. Willis, Jennifer Bitz and Allan Best
Implementation Science, 2013

Link to Article

Knowledge and Action for System Transformation (KAST): A Systematic Realist Review and Evidence Synthesis of the Role of Government Policy in Coordinating Large System Transformation

By Allan Best et al.
Centre for Clinical Epidemiology and Evaluation, Vancouver Coastal Health Research Institute, 2010

Large-System Transformation in Health Care: A Realist Review

By Allan Best, Trisha Greenhalgh, Steven Lewis, Jessie E. Saul, Simon Carroll and Jennifer Bitz
The Milbank Quarterly, 2012

Link to Article

Acknowledgments

Allan Best, Ph.D.

Managing Director, InSource

Systems Grantmaking Resource Guide

Resource Type

Narrative Reporting

Systems Questions

People

Variables

Patterns and Structure

Grantmaking Stages

Develop grantmaking strategy

Identify and select grantees

Participation

Not participatory

Expertise

Someone who has been trained on the resource

Time

1-5 months