Advancing the application of systems thinking in health: a realist evaluation of a capacity building programme for district managers in Tumkur, India

Health Research Policy and Systems

Bart Criel²  Guy Kegels²  Narayanan Devadasan¹  Bruno Marchal²  Nuggehalli Srinivas Prashanth² 

[1] Institute of Public Health, #250, 2 C Main, 2 C Cross, Girinagar I Phase, Bangalore 560 085, Karnataka, India;Institute of Tropical Medicine, Nationalestraat 155, 2000 Antwerp, Belgium

关键词: Programme theory;    Systems thinking;    Self-efficacy;    Realist evaluation;    Organisational commitment;    District health system;    Capacity building;   

Others  :  1177333
DOI  :  10.1186/1478-4505-12-42

 received in 2013-12-15, accepted in 2014-06-04,  发布年份 2014

【 摘 要 】

Background

Health systems interventions, such as capacity-building of health workers, are implemented across districts in order to improve performance of healthcare organisations. However, such interventions often work in some settings and not in others. Local health systems could be visualised as complex adaptive systems that respond variously to inputs of capacity building interventions, depending on their local conditions and several individual, institutional, and environmental factors. We aim at demonstrating how the realist evaluation approach advances complex systems thinking in healthcare evaluation by applying the approach to understand organisational change within local health systems in the Tumkur district of southern India.

Methods

We collected data on several input, process, and outcome measures of performance of the talukas (administrative sub-units of the district) and explore the interplay between the individual, institutional, and contextual factors in contributing to the outcomes using qualitative data (interview transcripts and observation notes) and quantitative measures of commitment, self-efficacy, and supervision style.

Results

The talukas of Tumkur district responded differently to the intervention. Their responses can be explained by the interactions between several individual, institutional, and environmental factors. In a taluka with committed staff and a positive intention to make changes, the intervention worked through aligning with existing opportunities from the decentralisation process to improve performance. However, commitment towards the organisation was neither crucial nor sufficient. Committed staff in two other talukas were unable to actualise their intentions to improve organisational performance. In yet another taluka, the leadership was able to compensate for the lack of commitment.

Conclusions

Capacity building of local health systems could work through aligning or countering existing relationships between internal (individual and organisational) and external (policy and socio-political environment) attributes of the organisation. At the design and implementation stage, intervention planners need to identify opportunities for such triggering alignments. Local health systems may differ in their internal configuration and hence capacity building programmes need to accommodate possibilities for change through different pathways. By a process of formulating and testing hypotheses, making critical comparisons, discovering empirical patterns, and monitoring their scope and extent, a realist evaluation enables a comprehensive assessment of system-wide change in health systems.

【 授权许可】

   
2014 Prashanth et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

Figure 7.

【 参考文献 】

• [1]LaFond AK, Brown L, Macintyre K: Mapping capacity in the health sector: a conceptual framework. Int J Health Plann Manage 2002, 17:3-22.
• [2]McDaniel RR, Lanham HJ, Anderson RA: Implications of complex adaptive systems theory for the design of research on health care organizations. Health Care Manage Rev 2009, 34:191-199.
• [3]Adam T, de Savigny D: Systems thinking for strengthening health systems in LMICs: need for a paradigm shift. Health Policy Plan 2012, 27(Suppl 4):iv1-iv3.
• [4]WHO: Working Together for Health: The World Health Report 2006. Geneva: World Health Organization; 2006:237.
• [5]Dieleman M, Gerretsen B, van der Wilt GJ: Human resource management interventions to improve health workers’ performance in low and middle income countries: a realist review. Heal Res policy Syst 2009, 7:7. BioMed Central Full Text
• [6]Mbindyo P, Gilson L, Blaauw D, English M: Contextual influences on health worker motivation in district hospitals in Kenya. Implement Sci 2009, 4:43. BioMed Central Full Text
• [7]Rowe A, Desavigny D, Lanata C, Victora C: How can we achieve and maintain high-quality performance of health workers in low-resource settings? Lancet 2005, 366:1026-1035.
• [8]Plsek PE, Greenhalgh T: Complexity science: the challenge of complexity in health care. Br Med J 2001, 323:625-628.
• [9]Potter C, Brough R: Systemic capacity building: a hierarchy of needs. Health Policy Plan 2004, 19:336-345.
• [10]Sturmberg JP, O’Halloran DM: Understanding health system reform – a complex adaptive systems perspective. J Eval Clin Pract 2012, 18:202-208.
• [11]Patton MQ: Developmental Evaluation: Applying Complexity Concepts to Enhance Innovation and Use. New York: Guilford Press; 2010.
• [12]Moore G, Audrey S, Barker M, Bond L, Bonell C, Cooper C, Hardeman W, Moore L, O’Cathain A, Tinati T, Wight D, Baird J: Process evaluation in complex public health intervention studies: the need for guidance. J Epidemiol Community Health 2014, 68(2):101-102.
• [13]Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M: Developing and evaluating complex interventions: the new Medical Research Council guidance. Br Med J 2008, 337:a1655-a1655.
• [14]Pawson R: The Science of Evaluation: A Realist Manifesto. 1st edition. London: Sage Publications; 2013:216.
• [15]Pawson R: Evidence-based policy: the promise of realist synthesis. Evaluation 2002, 8:340.
• [16]Wong G, Greenhalgh T, Westhorp G, Buckingham J, Pawson R: RAMESES publication standards: realist syntheses. BMC Med 2013, 11:21. BioMed Central Full Text
• [17]Pawson R, Tilley NN, Clarke A: Realistic Evaluation. 1st edition. London: Sage Publications Ltd.; 1997:256.
• [18]Bunge M: How does it work? The search for explanatory mechanisms. Philos Soc Sci 2004, 34:182-210.
• [19]Astbury B, Leeuw FL: Unpacking black boxes: mechanisms and theory building in evaluation. Am J Eval 2010, 31:363-381.
• [20]Pawson R, Tilley N: Realist evaluation. In DPRN Thematic Meeting 2006 Report on Evaluation. Utrecht, The Netherlands: Development policy review Network; 2008:35.
• [21]Jagosh J, Pluye P, Wong G, Cargo M, Salsberg J, Bush PL, Herbert CP, Green LW, Greenhalgh T, Macaulay AC: Critical reflections on realist review: insights from customizing the methodology to the needs of participatory research assessment. Res Synth Methods 2013, 5(2):131-141.
• [22]Marchal B: Why do Some Hospitals Perform Better than Others? A Realist Evaluation of the Role of Health Workforce Management in Well-Performing Health Care Organisations. Antwerp: Faculty of Medicine and Pharmacy, Vrije Universiteit Brussel & Institute of Tropical Medicine; 2011:227.
• [23]Kernick D: Complexity and Healthcare Organization: A View from the Street. 1st edition. London: Radcliffe Publishing Ltd.; 2004:400.
• [24]Prashanth NS, Marchal B, Criel B: Evaluating healthcare interventions: answering the “How” question. Indian Anthropol 2013, 43:35-50.
• [25]Marchal B, Van Belle S, De Brouwere V, Witter S: Studying complex interventions: reflections from the FEMHealth project on evaluating fee exemption policies in West Africa and Morocco. BMC Health Serv Res 2013, 13:469. BioMed Central Full Text
• [26]Marchal B, van Belle S, van Olmen J, Hoeree T, Kegels G: Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation 2012, 18:192-212.
• [27]Sheikh K: Unlocking the potential of qualitative enquiry into health policy and systems. In Second Global Symposium Health Systems Research. Beijing: World Health Organization; 2012:23. Available at http://hsr2012.healthsystemsresearch.org/images/stories/media/1102/3%20Kabir%20Sheikh.pdf webcite (Accessed June 17, 2014)
• [28]Westhorp G: Using complexity-consistent theory for evaluating complex systems. Evaluation 2012, 18:405-420.
• [29]Sicotte C, Champagne F, Contandriopoulos AP, Barnsley J, Béland F, Leggat SG, Denis JL, Bilodeau H, Langley A, Brémond M, Baker GR: A conceptual framework for the analysis of health care organization’s performance. Health Services Management Research 1998, 11(1):24-48. Retrieved from http://www.ncbi.nlm.nih.gov/pubmed/10178368 webcite
• [30]Marchal B, Hoerée T, da Silveira VC, Van Belle S, Prashanth NS, Kegels G: Building on the EGIPPS performance assessment: the multipolar framework as a heuristic to tackle the complexity of performance of public service oriented health care organisations. BMC Public Health 2014, 14:378. BioMed Central Full Text
• [31]Anil Kumar TK: Census of India 2011: provisional population totals paper 1 of 2011 India Series 1. New Delhi, India: Office of the Registrar General & Census Commissioner; 2011. Available at: http://censuskarnataka.gov.in/Provisional%20Population%20Totals webcite,%20Paper%201%20of%202011,%20Karnataka.pdf (Accessed 17 June 2014)
• [32]Government of Karnataka: Investing in Human Development: Karnataka Human Development Report 2005. Bangalore: Department of Planning and Statistics, GoK; 2005.
• [33]George A: Persistence of high maternal mortality in Koppal district, Karnataka, India: observed service delivery constraints. Reprod Health Matters 2007, 15:91-102.
• [34]Devadasan N, Elias MA: Training Needs Assessment for District Health Managers. Bangalore. Bangalore: Institute of Public Health; 2008:78. Available at https://docs.google.com/file/d/0Bxp4UKSObSs9ODlhNjQwNjctMGNlNC00MjhlLWJmY2QtMGVmN2I3YmY2YWE2 webcite
• [35]Sathyanarayan TN, Babu GR: Creating a public health cadre in India: the development of a framework for interprofessional and inter-sector collaboration. J Interprof Care 2011, 25:308-310.
• [36]Balabanova D, Mckee M, Mills A: Good Health at Low Cost 25 Years On. What Makes a Successful Health System. London: London School of Hygiene & Tropical Medicine; 2011:369.
• [37]Das GM, Desikachari BR, Shukla R, Somanathan TV, Padmanaban P, Datta KK: How might India’s public health systems be strengthened? Lessons from Tamil Nadu. Econ Polit Wkly 2010, xlv:46-60.
• [38]Hoeree T, Prasad V, Jiang L, Pongsupap Y: External Evaluation of the Tumkur Training Course. Bangalore: Institute of Public Health; 2012:50.
• [39]Prashanth NS, Marchal B, Hoeree T, Devadasan N, Macq J, Kegels G, Criel B: How does capacity building of health managers work? A realist evaluation study protocol. BMJ Open 2012, 2:e000882.
• [40]Lipsey MW, Pollard JA: Driving toward theory in program evaluation: more models to choose from. Eval Program Plann 1989, 12:317-328.
• [41]Prashanth NS, Marchal B, Kegels G, Criel B: Evaluation of a capacity-building programme of district health managers in india: a contextualised theoretical framework. Front. Public Health 2014, 2:89. doi: 10.3389/fpubh.2014.00089
• [42]Bandura A: Self-efficacy mechanism in human agency. Am Psychol 1982, 37:122-147.
• [43]Meyer JP, Paunonen SV, Gellatly IR, Goffin RD, Jackson DN: Organizational commitment and job performance: it’s the nature of the commitment that counts. J Appl Psychol 1989, 74:152-156.
• [44]Meyer JP, Allen NJ: A three-component conceptualization of organizational commitment. Hum Resour Manag Rev 1991, 1:61-89.
• [45]Bandura A: Guide for constructing self-efficacy scales. In Self-Efficacy Beliefs Adolesc. Edited by Pajares F, Urdan TC. Charlotte, NC: Information Age Publishing; 2006:307-337.
• [46]Oldham GR, Cummings A: Employee creativity: personal and contextual factors at work. Acad Manag J 1996, 39:607-634.
• [47]Cammann C, Fichman M, Jenkins G, Klesh J: The Michigan Organizational Assessment Package. Ann Arbor, MI: University of Michigan Survey Research Center; 1978.
• [48]Kirkpatrick DL, Kirkpatrick JD: Evaluating Training Programmes: The Four Levels. 2nd edition. San Francisco, CA: Berrett-Koehler Publishers, Inc.; 1998:289.
• [49]Clarke N: Workplace learning environment and its relationship with learning outcomes in healthcare organizations. Hum Resour Dev Int 2005, 8:185-205.
• [50]Marchal B, Dedzo M, Kegels G: A realist evaluation of the management of a well-performing regional hospital in Ghana. BMC Health Serv Res 2010, 10:24. BioMed Central Full Text
• [51]Government of Karnataka: Report of the High Power Committee for Redressal of Regional Imbalances. Bangalore: Government of Karnataka; 2004.
• [52]Marchal B, Dedzo M, Kegels G: Turning around an ailing district hospital: a realist evaluation of strategic changes at Ho Municipal Hospital (Ghana). BMC Public Health 2010, 10:787. BioMed Central Full Text
• [53]Parsons T: Social Systems and the Evolution of Action Theory. New York, NY: The Free Press; 1977.
• [54]Stame N: Theory-based evaluation and types of complexity. Evaluation 2004, 10:58.
• [55]De Souza DE: Elaborating the Context-Mechanism-Outcome configuration (CMOc) in realist evaluation: a critical realist perspective. Evaluation 2013, 19:141-154.
• [56]Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R: How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q 2009, 87:391-416.
• [57]Pratschke J: Realistic models? Critical realism and statistical models in the social sciences. Philosophica 2003, 71:13-39.