The concept of mechanism from a realist approach: a scoping review to facilitate its operationalization in public health program evaluation

Implementation Science

Valéry Ridde⁴  Anne Guichard³  Eric Breton¹  Anthony Lacouture² 

[1] CNRS, UMR CRAPE Centre for Research on Political Action in Europe—6051, Rennes, France;ESPUM School of Public Health University of Montreal, Montreal, Quebec, Canada;Faculty of Nursing, Laval University, Quebec, Quebec, Canada;IRSPUM University of Montreal Public Health Research Institute, Montreal, Quebec, Canada

关键词: Context;    Complexity;    Public health intervention;    Conceptual analysis;    Realism;    Evaluation;    Realist approach;    Mechanism;   

Others  :  1230928
DOI  :  10.1186/s13012-015-0345-7

 received in 2015-04-23, accepted in 2015-10-26,  发布年份 2015

【 摘 要 】

Background

Public health interventions are complex by nature, and their evaluation requires unpacking their intervention logic and their interactions with open social systems. By focusing on the interrelationships between context, mechanism, and outcome, Pawson and Tilley’s realist approach appears a promising innovation for public health-related evaluation works. However, and as expected of any methodological innovation, this approach is being constructed gradually by answering the multiple challenges to its operationalization that fall in its path. One of these challenges, users of this approach agree on, is the necessity of clarifying its key concept of mechanism.

Method

We first collected the definitions of mechanism from published works of Pawson and colleagues. Secondly, a scoping review was conducted to identify the ones quoted by users of the realist approach for evaluating public health interventions (1997–2012). We then appraised the clarity and precision of this concept against the three dimensions defined by Daigneault and Jacobs “term, sense and referent.”

Results

Of the 2344 documents identified in the scoping review, 49 documents were included. Term: Users of the realist approach use adjectives qualifying the term mechanism that were not specifically endorsed by Pawson and colleagues. Sense: None of the attributes stated by Pawson and colleagues has been listed in all of the documents analyzed, and some contributions clarified its attributes. Referent: The concept of mechanism within a realist approach can be ascribed to theory-based evaluation, complex social interventions, and critical realism.

Conclusion

This review led us to reconsider the concept of mechanism within the realist approach by confronting the theoretical stance of its proponents to the practical one of its users. This resulted in a clearer, more precise definition of the concept of mechanism which may in turn trigger further improvements in the way the realist approach is applied in evaluative practice in public health and potentially beyond. A mechanism is hidden but real, is an element of reasoning and reactions of agents in regard to the resources available in a given context to bring about changes through the implementation of an intervention, and evolves within an open space-time and social system of relationships.

【 授权许可】

   
2015 Lacouture et al.

【 图 表 】

Fig. 1.

Fig. 2.

Fig. 1.

Fig. 2.

【 参考文献 】

• [1]McQueen DV, Kickbusch I, Potvin L, Pelikan JM, Balbo L, Abel Th. Health and modernity: the role of theory in health promotion. New York: Springer; 2007
• [2]Petticrew M. When are complex interventions “complex”? When are simple interventions “simple”? Eur J Public Health. 2011; 21(4):397-8.
• [3]Connelly JB. Evaluating complex public health interventions: theory, methods and scope of realist enquiry. J Eval Clin Pract. 2007; 13(6):935-41.
• [4]Ridde V. Reducing social inequalities in health: public health, community health or health promotion? Promot Educ. 2007; 14(2):63-7.
• [5]Weiss CH. How can theory-based evaluation make greater headway? Eval Rev. 1997; 21(4):501-24.
• [6]Chen HT, Rossi PH. The multi-goal, theory-driven approach to evaluation: a model linking basic and applied social science. Soc Forces. 1980; 59(1):106-22.
• [7]Chen HT, Rossi PH. The theory-driven approach to validity. Eval Program Plann. 1987; 10(1):95-103.
• [8]Stame N. Theory-based evaluation and types of complexity. Evaluation. 2004; 10(1):58-76.
• [9]Pawson R, Tilley N. Realistic evaluation. Sage publications, London; 1997.
• [10]Rogers P. Using programme theory to evaluate complicated and complex aspects of interventions. Evaluation. 2008; 14(1):29-48.
• [11]Sridharan S, Nakaima A. Towards an evidence base of theory-driven evaluations: some questions for proponents of theory-driven evaluation. Evaluation. 2012; 18(3):378-95.
• [12]Pawson R, Tilley N. Realist evaluation. Cabinet Office, London; 2004.
• [13]Blaise P. Culture qualité et organisation bureaucratique, le défi du changement dans les systèmes publics de santé. Une évaluation réaliste de projets qualité en Afrique [thèse]. Université Libre de Bruxelles, Bruxelles; 2004.
• [14]Blaise P, Marchal B, Lefèvre P, Kegels G. Au-delà des méthodes expérimentales : l’approche réaliste en évaluation. In: Réduire les inégalités sociales en santé. Potvin L, Moquet MJ, Jones C, editors. INPES coll. Santé en action, Saint-Denis; 2010: p.285-96.
• [15]Astbury B, Leeuw FL. Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval. 2010; 31(3):363-81.
• [16]Marchal B, Van Belle S, Van Olmen J, Hoerée T, Kegels G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. 2012; 18(2):192-212.
• [17]Ridde V, Robert E, Guichard A, Blaise P, Van Olmen J. L’approche Realist à l’épreuve du réel de l’évaluation des programmes. Revue Canadienne d’évaluation de Programme. 2012; 26(3):14.
• [18]Marchal B, Dedzo M, Kegels G. A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Serv Res. 2010; 10:24. BioMed Central Full Text
• [19]Dalkin SM, Greenhalgh J, Jones D, Cunningham B, Lhussier M. What’s in a mechanism? Development of a key concept in realist evaluation. Implement Sci. 2015; 10:49. BioMed Central Full Text
• [20]Daigneault PM, Jacob S. Les concepts souffrent-ils de négligence bénigne en sciences sociales? Éléments d’analyse conceptuelle et examen exploratoire de la littérature francophone à caractère méthodologique. Soc Sci Inf. 2012; 51(2):188-204.
• [21]Mays N, Roberts E, Popay J. Synthesising research evidence. In: Studying the organization and delivery of health services: research methods. Fulop N, Allen P, Clarke A, Black N, editors. Routledge, London; 2001: p.188-220.
• [22]Arksey H, O’Malley L. Scoping studies: towards a methodological framework. Int J Soc Res Methodol. 2005; 8(1):19-32.
• [23]Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009; 6:6.
• [24]Pawson R. Evidence-based policy: the promise of “realist synthesis”. Evaluation. 2002; 8(3):340-58.
• [25]Pawson R. Evidence-based policy: a realist perspective. Sage, Thousand Oaks; 2006.
• [26]Pawson R, Greenhalgh T, Harvey G, Walshe K. Realist review—a new method of systematic review designed for complex policy interventions. J Health Serv Res Policy. 2005; 10:21-34.
• [27]Pawson R, Wong G, Owen L. Known knowns, known unknowns, unknown unknowns: the predicament of evidence-based policy. Am J Eval. 2011; 32(4):518-46.
• [28]Pawson R, Manzano-Santaella A. A realist diagnostic workshop. Evaluation. 2012; 18(2):176-91.
• [29]Blaise P, Kegels G. A realistic approach to the evaluation of the quality management movement in health care systems: a comparison between European and African contexts based on Mintzberg’s organizational models. Int J Health Plann Manage. 2004; 19(4):337-64.
• [30]Marchal B, Dedzo M, Kegels G. Turning around an ailing district hospital: a realist evaluation of strategic changes at Ho Municipal Hospital (Ghana). BMC Public Health. 2010; 10:787. BioMed Central Full Text
• [31]Ridde V, Guichard A. Perception De Quelques Mécanismes Favorables À La Réduction Des Inégalités Sociales De Santé En France. Glob Health Promot. 2011; 18(3):47-60.
• [32]Pommier J, Guével MR, Jourdan D. Evaluation of health promotion in schools: a realistic evaluation approach using mixed methods. BMC Public Health. 2010; 10:1-12. BioMed Central Full Text
• [33]Robert E, Ridde V, Marchal B, Fournier P. Protocol: a realist review of user fee exemption policies for health services in Africa. BMJ Open. 2012; 2(1):e000706.
• [34]Wand T, White K, Patching J. Applying a realist(ic) framework to the evaluation of a new model of emergency department based mental health nursing practice. Nurs Inq. 2010; 17(3):231-9.
• [35]Rycroft-Malone J, Wilkinson JE, Burton CR, Andrews G, Ariss S, Baker R et al.. Implementing health research through academic and clinical partnerships: a realistic evaluation of the Collaborations for Leadership in Applied Health Research and Care (CLAHRC). Implement Sci. 2011; 6(1):74-85. BioMed Central Full Text
• [36]Jagosh J, Pluye P, Macaulay AC, Salsberg J, Henderson J, Sirett E et al.. Assessing the outcomes of participatory research: protocol for identifying, selecting, appraising and synthesizing the literature for realist review. Implement Sci. 2011; 6(1):24-31. BioMed Central Full Text
• [37]Kastner M, Estey E, Perrier L, Graham ID, Grimshaw J, Straus SE et al.. Understanding the relationship between the perceived characteristics of clinical practice guidelines and their uptake: protocol for a realist review. Implement Sci. 2011; 6(1):69-77. BioMed Central Full Text
• [38]Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J et al.. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Q. 2012; 90:2.
• [39]Byng R, Norman I, Redfern S, Jones R. Exposing the key functions of a complex intervention for shared care in mental health: case study of a process evaluation. BMC Health Serv Res. 2008; 8:274. BioMed Central Full Text
• [40]Robert E. Etat des lieux des connaissances disponibles sur les politiques et les programmes d’exemption des paiements directs des soins en Afrique de l’Ouest (Bénin, Burkina Faso, Ghana, Guinée, Mali, Niger et Sénégal). Projet de capitalisation d’expériences d’exemption du paiement des soins en faveur des populations vulnérables du Sahel en Afrique de l’Ouest (CRCHUM, ONG HELP, ECHO); 2011.
• [41]Holma K, Kontinen T. Realistic evaluation as an avenue to learning for development NGOs. Evaluation. 2011; 17(2):181-92.
• [42]Clark AM, Whelan HK, Barbour R, MacIntyre PD. A realist study of the mechanisms of cardiac rehabilitation. J Adv Nurs. 2005; 52(4):362-71.
• [43]Pittam G, Boyce M, Secker J, Lockett H, Samele C. Employment advice in primary care: a realistic evaluation. Health Soc Care Community. 2010; 18(6):598-606.
• [44]Ranmuthugala G, Cunningham FC, Plumb JJ, Long J, Georgiou A, Westbrook JI et al.. A realist evaluation of the role of communities of practice in changing healthcare practice. Implement Sci. 2011; 6(1):49-54. BioMed Central Full Text
• [45]Byng R, Norman I, Redfern S. Using realistic evaluation to evaluate a practice-level intervention to improve primary healthcare for patients with long-term mental illness. Evaluation. 2005; 11(1):69-93.
• [46]Kane SS, Gerretsen B, Scherpbier R, Dal Poz M, Dieleman M. A realist synthesis of randomised control trials involving use of community health workers for delivering child health interventions in low and middle income countries. BMC Health Serv Res. 2010; 10:286-92. BioMed Central Full Text
• [47]Dieleman M, Gerretsen B, van der Wilt GJ. Human resource management interventions to improve health workers’ performance in low and middle income countries: a realist review. Health Res Policy Syst. 2009; 7:1-13. BioMed Central Full Text
• [48]Maluka S, Kamuzora P, SanSebastián M, Byskov J, Ndawi B, Olsen ØE et al.. Implementing accountability for reasonableness framework at district level in Tanzania: a realist evaluation. Implement Sci. 2011; 6:11. BioMed Central Full Text
• [49]Wand T, White K, Patching J. Realistic evaluation of an emergency department-based mental health nurse practitioner outpatient service in Australia. Nurs Health Sci. 2011; 13(2):199-206.
• [50]Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q. 2009; 87(2):391-416.
• [51]Steven A, Dickinson C, Pearson P. Practice-based interprofessional education: looking into the black box. J Interprof Care. 2007; 21(3):251-64.
• [52]Hoddinott P, Britten J, Pill R. Why do interventions work in some places and not others: a breastfeeding support group trial. Soc Sci Med. 2010; 70:769-78.
• [53]Wong G, Greenhalgh T, Westhorp G, Pawson R. Realist methods in medical education research: what are they and what can they contribute? Med Educ. 2012; 46(1):89-96.
• [54]Tolson D, McIntosh J, Loftus L, Cormie P. Developing a managed clinical network in palliative care: a realistic evaluation. Int J Nurs Stud. 2007; 44(2):183-95.
• [55]Pedersen LM, Nielsen KJ, Kines P. Realistic evaluation as a new way to design and evaluate occupational safety interventions. Saf Sci. 2012; 50(1):48-54.
• [56]Wilson V, McCormack B. Critical realism as emancipatory action: the case for realistic evaluation in practice development. Nurs Philos. 2006; 7(1):45-7.
• [57]Rycroft-Malone J, McCormack B, Hutchinson AM, DeCorby K, Bucknall TK, Kent B et al.. Realist synthesis: illustrating the method for implementation research. Implement Sci. 2012; 7(1):33. BioMed Central Full Text
• [58]Connelly JB, Duaso MJ, Butler G. A systematic review of controlled trials of interventions to prevent childhood obesity and overweight: a realistic synthesis of the evidence. Public Health (Elsevier). 2007; 121(7):510-7.
• [59]Lhussier M, Bangash S, Dykes F, Zaman M, Lowe NM. Development and implementation of a nutrition intervention programme in North West Pakistan: a realist framework. Health Promotion International. 2011; doi:10.1093/heapro/dar067.
• [60]Jackson SF, Kolla G. A new realistic evaluation analysis method: linked coding of context, mechanism, and outcome relationships. Am J Eval. 2012; 33(3):339-49.
• [61]Rycroft-Malone J, Fontenla M, Bick D, Seers K. A realistic evaluation: the case of protocol-based care. Implement Sci. 2010; 5:38. BioMed Central Full Text
• [62]Macfarlane F, Greenhalgh T, Humphrey C, Hughes J, Butler C, Pawson R. A new workforce in the making? A case study of strategic human resource management in a whole-system change effort in healthcare. J Health Organ Manag. 2011; 25(1):55-72.
• [63]Ogrinc G, Batalden P. Realist evaluation as a framework for the assessment of teaching about the improvement of care. J Nurs Educ. 2009; 48(12):661-7.
• [64]Oroviogoicoechea C, Watson R. A quantitative analysis of the impact of a computerised information system on nurses’ clinical practice using a realistic evaluation framework. Int J Med Inform. 2009; 78(12):839-49.
• [65]Macaulay AC, Jagosh J, Seller R, Henderson J, Cargo M, Greenhalgh T et al.. Assessing the benefits of participatory research: a rationale for a realist review. Glob Health Promot. 2011; 18(2):45-8.
• [66]Porter S, O’Halloran P. The use and limitation of realistic evaluation as a tool for evidence-based practice: a critical realist perspective. Nurs Inq. 2012; 19(1):18-28.
• [67]Rycroft-Malone J, Fontenla M, Bick D, Seers K. Protocol-based care: impact on roles and service delivery. J Eval Clin Pract. 2008; 14(5):867-73.
• [68]Evans D, Killoran A. Tackling health inequalities through partnership working: learning from a realistic evaluation. Crit Public Health. 2000; 10(2):125-40.
• [69]Ogilvie D, Bull F, Powell J, Cooper AR, Brand C, Mutrie N et al.. An applied ecological framework for evaluating infrastructure to promote walking and cycling: the iConnect study. Am J Public Health. 2011; 101(3):473-81.
• [70]Leone L. Realistic evaluation of an illicit drug deterrence programme: analysis of a case study. Evaluation. 2008; 14(1):9-28.
• [71]Keller C, Gare K, Edenius M, Lindblad S. Innovations in health care: design theory and realist evaluation combined. Sprouts: Working Papers on Information Systems. 2010; 10:66.
• [72]Kœnig G. Realistic Evaluation and Case Studies Stretching the Potential. Evaluation. 2009; 15(1):9-30.
• [73]Wong G, Pawson R, Owen L. Policy guidance on threats to legislative interventions in public health: a realist synthesis. BMC Public Health. 2011; 11(1):222-32. BioMed Central Full Text
• [74]McCormack B, Wright J, Dewar B, Harvey G, Ballantine K. A realist synthesis of evidence relating to practice development: methodology and methods. Pract Dev Health Care. 2007; 6(1):5-24.
• [75]Wong G, Greenhalgh T, Pawson R. Internet-based medical education: a realist review of what works, for whom and in what circumstances. BMC Med Educ. 2010; 10:12. BioMed Central Full Text
• [76]Michielsen J, Criel B, Devadasan N, Soors W, Wouters E, Meulemans H. Can health insurance improve access to quality care for the Indian poor? Int J Qual Health Care. 2011; 23(4):471-86.
• [77]Pawson R. The science of evaluation: a realist manifesto. SAGE Publications, London; 2013.
• [78]Bhaskar R. A realist theory of science. Revisedth ed. London: Routledge; 2008.
• [79]Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009; 43:267-76.
• [80]Hawe P, Shiell A, Riley T. Complex interventions: how “out of control” can a randomised controlled trial be? BMJ. 2004; 328:1561.
• [81]Archer M. Realist social theory: the morphogenetic approach. Cambridge University Press, Cambridge; 1995.
• [82]Poland B, Frohlich KL, Cargo M. Context as a fundamental dimension of Health Promotion Program Evaluation. In: Health promotion evaluation practices in the Americas: values and research. Potvin L, McQueen D, editors. Springer, New York; 2008: p.299-317.
• [83]Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R. RAMESES publication standards: realist syntheses. BMC Med. 2013; 11:21. BioMed Central Full Text
• [84]Greenhalgh T, Wong G, Jagosh J, Greenhalgh J, Manzano A, Westhorp G et al.. Protocol—the RAMESES II study: developing guidance and reporting standards for realist evaluation. BMJ Open. 2015; 5:8.