Such reviews take stock of the research available in a field of study, such that one can decide how to develop the field further (Arksey & O’Malley, 2005; Armstrong, Hall, Doyle, & Waters, 2011; Bates, Clapton, & Coren, 2007; Clapton, Rutter, D., & Sharif, 2009; Oakley, Gough, Oliver, & James, 2005; Shepherd et al., 2006). Poster presentation at Cochrane Colloquium, Singapore. studies. The first four items of the RAMESES Publication Standards deal with the information expected for the title, abstract, and introduction of a realist review (Items 1-4). It is peer reviewed and is currently indexed in International Bibliography of the Social Sciences, ProQuest Academic Research Library, ProQuest Central, ProQuest Social Sciences, The Philosophers Index and PhilPapers. Our review of what approaches to social prescribing work, for whom, and in what circumstances, led by Kerryn Husk at the University of Plymouth and Ruth Garside here at ECEHH, has been published in the journal Health and Social Care in the Community!. To read the fulltext, please use one of the options below to sign in or purchase access. (2016). It will not tell policy-makers or managers whether something works or not, but will provide the policy and practice community with the kind of rich, detailed and highly practical understanding of complex social interventions which is likely to be of much more use to them when planning and implementing programmes at a national, regional or local level. Analysis and synthesis process: (a) Describe the analysis process in detail. Szifris et al./Journal of Prison Education and Reentry 5(1) 43 both for prison education and future research. Counter to the publication standards and the review developers’ expectation that realist reviews are particularly oriented towards practitioners and policy makers (Pawson, 2006; Pawson et al., 2004, 2005), 83 per cent of the reviews did not specify the main implications of their findings and 67 per cent did not state whether they had any policy linkage. Therefore, we conducted a review of the published examples of realist review. Contact us if you experience any difficulty logging in. Table 1 shows the descriptive results for each reporting item to be included when reporting a realist review, Table 1. These results indicate a relatively low degree of similarity among those reviews meeting our broad criteria for realist review, and in particular great variability in reporting of methods in realist reviews. Greenhalgh, T., Wong, G., Westhorp, G., & Pawson, R. (2011). A three-pronged approach was used incorporating an initial search (6), a database search using applicable keywords and MeSH headings (58), and review of literature identified by advisory group (8 grey literature). Department of Sociology and Social Policy, University of Leeds, Leeds UK. Flow diagram of the search and selection process. The time-frame to conduct this review was short; it was the preliminary stage of a realist evaluation which was also time-limited. Overall, there was little uniformity and transparency regarding many methodological issues. The method is especially useful for synthesizing methodologically diverse empirical studies. International Journal of Social Research Methodology, 8(1), 19-32. Evidence-based policy: A realist perspective. British Journal of Music Therapy, Vol. ESRC Research Methods Programme, University of Manchester, UK. The results of this examination are summarized in Table 2. You can be signed in via any or all of the methods shown below at the same time. (2007). It is not used to evaluate specific interventions and/or programmes, rather it synthesizes existing evalua-tions. (2013). Realist reviews follow most of the same standard steps as systematic reviews (Pawson et al., 2004): Pawson and colleagues (2004) stipulate that the process should be equally rigorous and transparent (i.e., auditable), where every judgment is written down. Information that could not be dichotomized is not included in the table, but is described in the following text. In 13 per cent of reviews, appraisal was done by two or more people. As shown in Table 2 (Row 3), less than a quarter (22 per cent) of the reviews stated that the process of selecting documents was done by two or more persons independently. Conducting a systematic review and meta-analysis in line with a realist review will allow us to draw a robust conclusion on the effects and the way in which the interventions work. Realist review is an interpretive, theory-driven narrative summary which applies realist philosophy of exploring context, mechanisms and outcomes . Realist review has emerged as a specific literature review approach that is concerned with explaining the outcomes of complex intervention programs. We have not extracted data from these abstracts. Members of _ can log in with their society credentials below, Journal of Health Services Research & Policy. Methods This is a realist review with iterative searches. Figure 1. Methodological concerns and quality appraisal of contemporary systematic reviews and meta-analyses in pediatric urology. Retrieved from Contain brief details of: background, review question/objectives, search strategy, methods of selection, appraisal, analysis, and synthesis of sources, main results, implications for practice, 3. A realist review method considers the larger context of the concepts, constructs, motivations, and underlying theory that are shaping the research articles [36]. Specifically, one stated, the “re-review offers a fuller understanding of the impacts of these interventions and how they are produced . Reporting of the RAMESES Publication Standards Items in the 54 Realist Reviews, 1. Successfully implementing a national electronic health record: a rapid... Searching for new community engagement approaches in the Netherlands: ... Greenwood, J. Realism, Identity and Emotion. The Journal of Prosthetic Dentistry, 111(6), 476-484. Instead, the reviews appeared to rely largely on a qualitative assessment of the likelihood of certain outcomes. Related to this, 10 reviews had no information about the years searched, while 8 had no limits for the year of publication (9 per cent), and the remaining 28 reviews searched literature published after a specified year. Journal of Clinical Epidemiology, 59, 697-703. Cochrane Reviewers, Handbook 4.2.0. Understanding how appraisal of doctors produces its effects: A realist review protocol, The application of realist synthesis review methods in public health economics, Improving palliative care in dementia and cancer: Organisation of palliative care-study protocol, Psychiatric advance directives: From a single document to a complex and multistage intervention, realist review, A conceptual framework for integrated palliative care interventions: Understanding when pathways work, don't work, and why, Supporting families affected by stroke: A realist synthesis of the intervention literature, Enhancing the uptake of clinical practice guidelines: The development of a guideline implementability tool (GUIDE-IT), Integration through engagement: A movement towards older adults' involvement in health care research, planning and decision making, International Journal of Integrative Care, 14 (Annual Conference Supplement), A realist systematic review of screening programs for intimate partner violence in health care settings, American Journal of Epidemiology, 169, S108, Mechanisms of an occupation-based approach for persons with chronic pain and work loss, School of Occupational Therapy, Western University, London, Canada, Research supports the need to develop a guideline to enhance work outcomes for injured workers, School of Social Work, University of Central Lancashire, Preston, UK, Case management's effectiveness to yield positive outcomes for frail older persons living at home: A realist review, International Journal of Integrative Care, 14(Annual Conference Supplement), Appendix C: Systematic review, realist synthesis and economic modelling, Understanding how self-management interventions work for disadvantaged populations living with chronic conditions: Protocol for a realist synthesis, A realist review of educational interventions to improve the delivery of nutrition care by doctors and future doctors, Protocol: Realist synthesis of the impact of unemployment insurance policies on poverty and health, Implementing health promotion in schools: Protocol for a realist systematic review of research and experience in the United Kingdom (UK), Protocol: A realist review of user fee exemption policies for health services in Africa, Exploring why quality circles work in primary health care: A realist review protocol, Improving skills and care standards in the support workforce for older people: A realist review, Does therapeutic writing help people with long term conditions? This product could help you, Accessing resources off campus can be a challenge. As Pawson has described elsewhere , key stages and tasks for a realist review include the following steps: identifying the review question, searching for primary studies, quality appraisal, extracting the data, synthesizing the data, and disseminating the findings (see , chapter 4). It should be noted, however, that provisional results and conclu-sions are common when examining mechanisms, since a For example, three quarters (74 per cent) of the 54 reviews identified the document as a realist review or realist synthesis in the title. Inclusion criteria for studies that addressed outcomes were: studies had to be controlled studies (n=5), evaluation studies (n=3), any type of quantitative studies, quantitative and qualitative studies that evaluated any of the outcomes, or studies that report on one or more of the outcomes of interest. a realist review does not provide concrete and absolute answers and, as a result, the possibility for additional contributions to each theory may remain, as findings from a realist review are often provisional [48]. Oakley, A., Gough, D., Oliver, S., & James, T. (2005). Index Terms: systematic review; systematic scoping review; mapping review; social intervention; realist review; realist synthesis; RAMESES publication standards, Suggested Citation: Berg, R. C., & Nanavati, J. Young people and healthy eating: A systematic review of research on barriers and facilitators. Funding: (a) Provide details of funding source (if any) for the review, 19. The methodological quality of empirical studies was assessed using the appropriate appraisal tool from the Critical Appraisal Skills Programme, 31 and studies were classified as strong, moderate, or weak in terms of rigor. Delegating home visits in general practice: a realist review on the impact on GP workload and patient care Ruth Abrams , et al British Journal of General Practice 1 June 2020; 70 (695): e412-e420. Note. Wolfsfeld, G. Media and Political Conflict. It is edited by Mervyn Hartwig and a team of other scholars. View or download all the content the society has access to. Disagreements were resolved by re-examination of the review and subsequent discussion. (1997). Cochrane handbook for systematic reviews of interventions (Version 5.1.0). Realist literature reviewing is an explanatory method that aims to examine how complex phenomena work, and in what contexts they occur. Only seven of the reviews included in our study, all published in 2014 or 2015, cited and stated that they used the RAMESES publication standards. Improving best practice for patients receiving hospital discharge letters: a realist review Katharine Weetman , Geoff Wong , Emma Scott , Eilidh MacKenzie , Stephanie Schnurr , Jeremy Dale BMJ Open Jun 2019, 9 (6) e027588; DOI: 10.1136/bmjopen-2018-027588 We considered 159 reviews in full text (Figure 1). We are not arguing that realist reviews should adopt the processes that are standard for systematic reviews. Similarly, the specific databases searched varied, although the most commonly searched were MEDLINE (n=30), PsycINFO (n=25), CINAHL (n=23), EMBASE (n=22), and the Cochrane Library (n=18). Note. Dixon, E., Hameed, M., Sutherland, F., Cook, D. J., & Doig, C. (2005). A Theory-Driven Systematic Review Working Paper No. Increasingly, practitioners, policy-makers, and others call for reviews that accommodate complexity and context in drawing lessons from complex intervention programs (Bravata et al., 2005). A Research Based Evaluation of the Orthopaedic Services Collaborative Within the Eastern, South and West, South East and Trent Regions. Littell, J. It is edited by Mervyn Hartwig and a team of other scholars. In accordance with our aim, we examined the description of methods and processes in the reviews, with respect to quality assurance and transparency. On the other hand, that more than half of the reviews we studied made no reference to scoping of the literature cannot be explained by imprecise reporting standards, and seems a major oversight as exploratory background searching is highlighted as an important first step in the key realist review texts (Pawson, 2006; Pawson et al., 2004, 2005). A good example is one that addressed intercultural doctor-patient communication (Paternotte, Van Dulmen, Van Der Lee, Scherpbier, & Scheele, 2015). We included reviews with a methodological focus, for example, the presentation of the realist review approach, if the authors reported on the findings of a particular review. Unleashing their potential: A critical realist scoping review of the influence of dogs on physical activity for dog-owners and non-owners. Thus realism does involve being realistic. Inter-reviewer agreement: An analysis of the degree to which agreement occurs when using tools for the appraisal, extraction and meta-synthesis of qualitative research findings. Journal of Further and Higher Education, v40 n3 p316-330 2016. Three reviews made no reference to how the documents were identified. We found that 14 reviews specified eligibility for studies that addressed the outcome of an intervention, one review highlighted criteria for mechanisms (views) papers, and one did so for context papers. One of the limitations is that new realist reviews are being published quite regularly in a variety of areas (sometimes without specifying its realist nature), and some may have been missed in this review. Higgins, J. P. T., & Green, S. (2011, March). Concept mapping in policy evaluation: a research review of community relations in Northern Ireland. Provide details on all the sources accessed for information in the review [d], 9. [b] Typically, eligibility criteria in evidence-based practice is specified by the acronym, PICO, which stands for Population, Intervention, Comparison, and Outcome. While full Realist reviews engage in a much longer exploration of the literature and period of ‘testing’, rapid realist reviews (RRRs) have recently emerged to facilitate a speedier transition from research to policy and practice. Armstrong, R., Hall, B. J., Doyle, J., & Waters, E. (2011). As an additional example, many reviews included vague inclusion questions such as: Does the article/document focus on [topic]? Realist review is a relatively new strategy for synthesiz- For example, two of the seven reviews stating that they followed the standards did not identify the review as a realist review in the title (RAMESES Item 1) and most did not report on the items regarding strengths, limitations, and future research directions (Item 16) or funding (Item 19). Funding: (c) Provide details of any conflict of interest of the reviewers, Notes. When methods are not sufficiently justified or are poorly undertaken or reported, and the approach subsequently appears to exist in many variants, it becomes harder to understand what is meant when the term realist review is used. The IACR journal is the Journal of Critical Realism (JCR). Realist review aims to answer “what works for whom in what circumstances, in what respects and how?” with respect to social intervention programs (Pawson et al., 2004, p. 3). 1 partially It was also challenging to designate some of the data points. Related to the RAMESES Publication Standard Item 8 (pertaining to the searching process), we extracted data about the methods used for searching for literature (i.e., the methods used to identify the documents to be considered for inclusion in the review). Connell, J , Kubish, A , Schorr, L , Weiss, C. New Approaches to Evaluating Community Initiatives. We will conduct a realist review of studies describing programs that address the transition to community from incarceration for people with HIV. Abstracts and protocols (a protocol is a document that describes the plan for conducting a study, including purpose and methods) of planned or ongoing reviews that appeared to meet the inclusion criteria were included on a separate list, but data from these were not extracted. We used Unclear for issues that could not be answered based on the information available in the publication. The realist understanding of how programmes work Realist philosophy (Pawson and Tilley use the term ‘scientific realism’) considers that an intervention works (or not) because actors make particular decisions in response to the intervention (or not). Annals of Surgery, 241(3), 450-459. Sackett, D. L., Richardson, W. S., Rosenberg, W., & Haynes, R. B. In our study, a high proportion of reviewers discussed the limitations of their review rather than the strengths, oftentimes highlighting the demanding methodological process. [c] A lack of reference to changes may not indicate that no changes were made; therefore, we only noted when such changes were explicitly stated Does the article/document make conclusion regarding the effectiveness of [intervention]? Social Science Research and Decision-Making. Similarly, without theoretical or methodological justifications, realist reviews may also be suspected of offering a partial and potentially misleading account of the evidence. Realist Review’s Pandemic Reading List. A Realist Review of Persuasive System Design Components in Internet-Based Psychological Therapies, Journal of Medical Internet Research, 10.2196/jmir.7573, 19, … operating at different levels, the artefacts they use and the material environments in which they work [ 17 ]. Lastly, we note that only a third of the reviews mentioned involving stakeholders at any point in the review process. This is not unique to realist reviews, but also identified in systematic mapping reviews of qualitative evidence syntheses in general (Dixon-Woods, Booth, & Sutton, 2007), as well as in meta-ethnographies in particular (France et al., 2014). We compared our assessments and obtained full text copies of the reviews deemed relevant. Greenhalgh, T , Robert, G , Bate, P , Kyriakidou, O , Macfarlane, F , Peacock, R. How to Spread Ideas: A Systematic Review of the Literature on Diffusion, Dissemination and Sustainability of Innovations in Health Service Delivery and Organisation. Journal of Research Practice, 12(1), Article R1. Details of 71 Publications Included in This Study, Understanding the effectiveness of school-based interventions to prevent suicide: A realist review, Child and Adolescent Mental Health, 18(3), 131-139, Interventions to reduce the prevalence of female genital mutilation/cutting in African countries, A realist synthesis of controlled studies to determine the effectiveness of interventions to prevent genital cutting of girls, Paediatrics and International Child Health, 33(4), Large-system transformation in health care: A realist review, A realist review of art Therapy for clients with depression, Towards a framework for enhancing the performance of NHS boards: A synthesis of the evidence about board governance, board effectiveness and board development, Health Service Delivery Research, 1(6), 1-158. International Journal of Behavioral Nutrition and Physical Activity, 8, 46, Improving organizational capacity to address health literacy in public health: A rapid realist review, Internet-based medical education: A realist (2009). [e] Four reviews could not be assessed with regard to this item because these were re-reviews of completed systematic reviews and used all or a subset of the studies included in the completed systematic review Braga, L. H., Pemberton, L., DeMaria, J., & Lorenzo, A. J. Access to society journal content varies across our titles. The final section of the RAMESES Publication Standards is labeled Discussion and includes five items (Items 15-19). Click the button below for the full-text content, 24 hours online access to download content. Realist review does not provide simple answers to complex questions. Four of the realist reviews were re-reviews of completed systematic reviews, which used all or a subset of the studies originally included in the systematic review. The e-mail addresses that you supply to use this service will not be used for any other purpose without your consent. A realist review method generated a total of 72 papers that were used to inform the results. Thus it is disquieting that only about one in five of the reviews in our study had selection, appraisal, or data extraction done independently by two persons. Although the number of publications reporting a realist review appears to be increasing rapidly, there is a lack of clarity about its methods and applications. Pawson, R. A Measure for Measures: A Manifesto for Empirical Sociology. Review Article. [h] Ten reviews could not be assessed with regard to this item because the reviews were not funded. Appendix A: Dixon-Woods, M., Booth, A., & Sutton, A. J. A realist review was conducted to research the way in which context influences how OSSs work. A., & Scheele, F. (2015). Arksey, H., & O’Malley, L. (2005). With respect to realist reviews, “experts still differ on detailed conceptual methodological issues” (Wong et al., 2013, p. 3). Part of the problem is one of complexity. We have not extracted data from these protocols. The review followed a realist, theory-driven approach to synthesizing qualitative, quantitative and mixed-methods literature. Key realist review texts (Pawson, 2006; Pawson et al., 2004, 2005; Wong et al., 2013) explain that appraisal of the data should be made on relevance (whether it can contribute to theory building and/or testing) and rigor (whether it is credible and trustworthy), which was the case for only a quarter of the reviews included in our study. Login failed. Beyond general systematic reviews, the authors selected a realist review for synthesizing the literature. Reporting of Quality Assurance and Transparency Measures in the 54 Realist Reviews, Quality Assurance and Realist review: Current practice and future prospects. Pawson, R., Greenhalgh, T., Harvey, G., & Walshe, K. (2005). In this realist review, we conceptualise educational interventions and strategies to improve the delivery of nutrition care as complex ones [ 18 ], which involve multiple actors (teachers, learners, patients, health care providers, etc.) Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & The PRISMA Group. The review was conducted in two phases: theory building and theory refinement. As with all ratings, there is some degree of subjectivity and other researchers may classify the information differently. Levac, D., Colquhoun, H., & O’Brien, K. K. (2010). Our approach and findings are presented below. Faculty of Health Sciences, University of Tromso, Tromso, NORWAY Data extraction: Describe and explain which data or information were extracted from the included documents and justify this selection [d], 11. We extracted data on these 19 items. In contrast to some effectiveness reviews, one review only accepted randomized controlled trials (RCT). Acting independently and jointly, we then assessed whether the reviews read in full text met all inclusion criteria. By continuing to browse Strengths, limitations, and future research directions: a) Discuss the strengths of the review, 16. Realist Review: Current Practice and Future Prospects, Rigmor C. Berg