New directions in evidence-based policy research: a critical analysis of the literature

Health Research Policy and Systems

Simon Innvær²  Theo Lorenc¹  Kathryn Oliver¹ 

[1] Department of Science, Technology, Engineering and Public Policy (STEaPP), University College London, 66-72 Gower Street, London WC1E 6BT, UK;Faculty of Social Sciences, Oslo University College, P.O Box 1084, Blindern, 0317 OSLO, Norway

关键词: Science and technology studies;    Knowledge utilization;    Evidence-based policy;    Critical analysis;   

Others  :  1181912
DOI  :  10.1186/1478-4505-12-34

 received in 2014-03-17, accepted in 2014-06-30,  发布年份 2014

【 摘 要 】

Despite 40 years of research into evidence-based policy (EBP) and a continued drive from both policymakers and researchers to increase research uptake in policy, barriers to the use of evidence are persistently identified in the literature. However, it is not clear what explains this persistence – whether they represent real factors, or if they are artefacts of approaches used to study EBP. Based on an updated review, this paper analyses this literature to explain persistent barriers and facilitators. We critically describe the literature in terms of its theoretical underpinnings, definitions of ‘evidence’, methods, and underlying assumptions of research in the field, and aim to illuminate the EBP discourse by comparison with approaches from other fields. Much of the research in this area is theoretically naive, focusing primarily on the uptake of research evidence as opposed to evidence defined more broadly, and privileging academics’ research priorities over those of policymakers. Little empirical data analysing the processes or impact of evidence use in policy is available to inform researchers or decision-makers. EBP research often assumes that policymakers do not use evidence and that more evidence – meaning research evidence – use would benefit policymakers and populations. We argue that these assumptions are unsupported, biasing much of EBP research. The agenda of ‘getting evidence into policy’ has side-lined the empirical description and analysis of how research and policy actually interact in vivo. Rather than asking how research evidence can be made more influential, academics should aim to understand what influences and constitutes policy, and produce more critically and theoretically informed studies of decision-making. We question the main assumptions made by EBP researchers, explore the implications of doing so, and propose new directions for EBP research, and health policy.

【 授权许可】

   
2014 Oliver et al.; licensee BioMed Central Ltd.

【 参考文献 】

• [1]Cooksey D: A Review of UK Health Research Funding. London: Stationery Office; 2006.
• [2]Fox DM: Evidence of evidence-based health policy: the politics of systematic reviews in coverage decisions. Health Affairs 2005, 24:114-122.
• [3]Mitton C, Adair C, McKenzie E, Patten S: Knowledge transfer and exchange: review and synthesis of the literature. Milbank Q 2007, 85:729-768.
• [4]Cochrane A: Effectiveness and Efficiency: Random Reflections on Health Services. London: Nuffield Provincial Hospitals Trust; 1972.
• [5]Champagne F, Lemieux-Charles L: Using Knowledge and Evidence in Health Care: Multidisciplinary Perspectives. Toronto: University of Toronto Press; 2008.
• [6]Guyatt G, Cairns J, Churchill D: Evidence-based medicine: a new approach to teaching the practice of medicine. JAMA 1992, 268:2420-2425.
• [7]Barton S: Which clinical studies provide the best evidence? BMJ 2000, 321(7256):255-256.
• [8]Chalmers I: If evidence-informed policy works in practice, does it matter if it doesn’t work in theory? Evidence Policy 2005, 1:227-242.
• [9]Glasziou PP, Irwig LM: An evidence based approach to individualising treatment. BMJ 1995, 311:1356-1359.
• [10]Guyatt GH: Users’ guides to the medical literature. JAMA 1995, 274:1800.
• [11]Kogan M, Henkel M, Britain G: Government and Research: The Rothschild Experiment in a Government Department. Portsmouth, NH: Heinemann Educational Books; 1983.
• [12]Hanney SR, González-Block MA: Evidence-informed health policy: are we beginning to get there at last. Health Res Policy Syst 2009, 7:30.
• [13]Duffy MP: The Rothschild experience: health science policy and society in Britain. Sci Tech Hum Val 1986, 11:68-78.
• [14]Government Office for Science: Engaging with Government: Guide for Academics, BIS/11/1360. 2011. [https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/283118/11-1360-guide-engaging-with-government-for-academics.pdf webcite]
• [15]Government Office for Science: Engaging with Academics: How to Further Strengthen Open Policy Making – A Guide For Policy Makers, BIS/13/581. 2013. [https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/283129/13-581-engaging-with-academics-open-policy-making.pdf webcite]
• [16]Martin BR: The Research Excellence Framework and the ‘impact agenda’: are we creating a Frankenstein monster? Res Eval 2011, 20:247-254.
• [17]Pettigrew AM: Scholarship with impact. Br J Manag 2011, 22:347-354.
• [18]HEFCE: Joint Statement on Impact by HEFCE, RCUK and UUK. 2011. [http://www.hefce.ac.uk/news/newsarchive/2011/news62296.html webcite]
• [19]Armstrong R, Waters E, Roberts H, Oliver S, Popay J: The role and theoretical evolution of knowledge translation and exchange in public health. J Public Health 2006, 28:384-389.
• [20]Dobbins M, Robeson P, Ciliska D, Hanna S, Cameron R, O'Mara L, DeCorby K, Mercer S: A description of a knowledge broker role implemented as part of a randomized controlled trial evaluating three knowledge translation strategies. Implement Sci 2009, 4:23.
• [21]Ward V, House A, Hamer S: Knowledge brokering: exploring the process of transferring knowledge into action. BMC Health Serv Res 2009, 9:12.
• [22]Cabinet O, Treasury HM, Open Public S, Rt Hon D, Alexander MP, Rt Hon O, Letwin MP: What Works: Evidence Centres for Social Policy. London: HM Government; 2013.
• [23]Walshe KWTR: Evidence-based management: from theory to practice in health care. Milbank Q 2001, 79:429-457.
• [24]Marston G, Watts R: Tampering with the evidence: a critical appraisal of evidence-based policy-making. The Drawing Board: Aust Rev Pub Affairs 2003, 3:143-163.
• [25]Black N, Donald A: Evidence based policy: proceed with care commentary: research must be taken seriously. BMJ 2001, 323:275-279.
• [26]Lavis J, Robertson D, Woodside J, McLeod C, Abelson J: How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003, 81:221-248.
• [27]Muir Gray J: Evidence-based policy making: is about taking decision based on evidence and the needs and values of the population. BMJ 2004, 329:988-989.
• [28]Nutbeam D: Getting evidence into policy and practice to address health inequalities. Health Promot Int 2004, 19:137-140.
• [29]Smith G: How policy informs the evidence. BMJ 2001, 322:184-185.
• [30]Oliver K, Innvaer S, Lorenc T, Woodman J, Thomas J: A systematic review of barriers to and facilitators of the use of evidence by policymakers. BMC Health Serv Res 2014, 14:2.
• [31]Orton L, Lloyd-Williams F, Taylor-Robinson D, O'Flaherty M, Capewell S: The use of research evidence in public health decision making processes: systematic review. PLoS ONE 2011, 6:e21704.
• [32]Boaz A, Baeza J, Fraser A, European Implementation Score Collaborative Group (EIS): Effective implementation of research into practice: an overview of systematic reviews of the health literature. BMC Res Notes 2011, 4:212.
• [33]Pawson R: Evidence-based Policy: A Realist Perspective. Thousand Oaks, CA: Sage Publications Ltd; 2006.
• [34]Innvaer S, Vist G, Trommald M, Oxman A: Health policy-makers’ perceptions of their use of evidence: a systematic review. J Health Serv Res Policy 2002, 7:239-244.
• [35]Lorenc T, Tyner E, Petticrew M, Duffy S, Martineau F, Philips G, Lock K: Cultures of evidence across policy sectors: systematic review of qualitative evidence. Eur J Public Health 2014. doi:10.1093/eurpub/cku038
• [36]Weiss CH: The many meanings of research utilization. Publ Admin Rev 1979, 39:426-431.
• [37]Weiss CH, Bucuvalas MJ: Social Science Research and Decision-Making. New York: Columbia University Press; 1980.
• [38]Green LW, Ottoson JM, Garcia C, Hiatt RA: Diffusion theory and knowledge dissemination, utilization, and integration in public health. Annu Rev Public Health 2009, 30:151-174.
• [39]Hanney S, Gonzalez-Block M, Buxton M, Kogan M: The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Pol Syst 2003, 1:2.
• [40]Sabatier PA, Jenkins-Smith H: Policy Change and Learning: An Advocacy Coalition Approach. San Francisco: Westview Press; 1993.
• [41]Dror Y: Public Policy Making Re-Examined. New Jersey: Transaction Publishers; 1983.
• [42]Aaserud M, Lewin S, Innvaer S, Paulsen E, Dahlgren A, Trommald M, Duley L, Zwarenstein M, Oxman AD: Translating research into policy and practice in developing countries: a case study of magnesium sulphate for pre-eclampsia. BMC Health Serv Res 2005, 5:68.
• [43]Hivon ML: Use of health technology assessment in decision making: coresponsibility of users and producers? Int J Tech Assess Health Care 2005, 21:268-275.
• [44]Jack SM: Knowledge transfer and exchange processes for environmental health issues in Canadian Aboriginal communities. Int J Environ Res Publ Health 2010, 7:651-674.
• [45]Priest N: Engaging policy makers in action on socially determined health inequities: developing evidence-informed cameos. Evid Policy 2009, 5:53-70.
• [46]Vingilis E, Hartford K, Schrecker T, Mitchell B, Lent B, Bishop J: Integrating knowledge generation with knowledge diffusion and utilization: a case study analysis of the Consortium for Applied Research and Evaluation in Mental Health. Canadian J Pub Health 2003, 94:468-471.
• [47]Sebba J: Enhancing impact on policy-making through increasing user engagement in research. In Educational Research and Policy-Making: Exploring the Border Country between Research and Policy. London & NY: Routledge; 2007:127-143.
• [48]Sabatier PA: Theories of the Policy Process. Boulder, CO: Westview Press; 2007.
• [49]Shore C, Wright S, Per D: Policy Worlds: Anthropology and Analysis of Contemporary Power. Oxford: Berghahn; 2011.
• [50]Kingdon JW: Agendas, Alternatives, and Public Policies. 3rd edition. Boston: Little, Brown; 2010.
• [51]Weiss CH: Nothing as practical as good theory: exploring theory-based evaluation for comprehensive community initiatives for children and families. In New Approaches to Evaluating Community Initiatives: Concepts, Methods, and Contexts. Queenstown, MD: The Aspen Institute; 1995:65-92.
• [52]Simon HA: Models of Man; Social and Rational. New York: John Wiley and Sons, Inc.; 1957.
• [53]Lindblom CE: The science of “muddling through”. Publ Admin Rev 1959, 19:79.
• [54]True JL, Jones BD, Baumgartner FR: Punctuated-equilibrium theory: explaining stability and change in American policymaking. In Theories of the Policy Process. Colorado: Westview Press Boulder; 1999:97-115.
• [55]Simon HA: Models of Bounded Rationality: Empirically Grounded Economic Reason. 3rd edition. Cambridge, MA: MIT press; 1982.
• [56]Lomas J, Brown A: Research and advice giving: a functional view of evidence-informed policy advice in a Canadian ministry of health. Milbank Q 2009, 87:903-926.
• [57]Cairney P: How can policy theory have an impact on policy making? The role of theory-led academic-practitioner discussions. Teach Pub Adminis 2014. doi:10.1177/0144739414532284
• [58]Buse K, Mays N, Walt G: Making Health Policy. New York: McGraw-Hill International; 2012.
• [59]Stevens A: Telling policy stories: an ethnographic study of the use of evidence in policy-making in the UK. J Soc Policy 2011, 40:237-255.
• [60]Burris H, Parkhurst J, Du-Sarkodie Y, Mayaud P: Getting research into policy - Herpes simplex virus type-2 (HSV-2) treatment and HIV infection: international guidelines formulation and the case of Ghana. Health Res Pol Syst 2011, 9(Suppl 1):S5.
• [61]Smith K, Joyce K: Capturing complex realities: understanding efforts to achieve evidence-based policy and practice in public health. Evid Policy 2012, 8:57-78.
• [62]Weitkamp G, Van den Berg AE, Bregt AK, Van Lammeren RJA: Evaluation by policy makers of a procedure to describe perceived landscape openness. J Environ Manage 2012, 95:17-28.
• [63]Aoki-Suzuki C, Bengtsson M, Hotta Y: International comparison and suggestions for capacity development in industrializing countries. J Ind Ecol 2012, 16:467-480.
• [64]Cerveny LK, Blahna DJ, Stern MJ, Mortimer MJ, Predmore SA, Freeman J: The use of recreation planning tools in U.S. Forest Service NEPA assessments. Environ Manag 2011, 48:644-657.
• [65]Jenkins RAR: Bridging data and decision making: development of techniques for improving the HIV prevention community planning process. AIDS Behav 2005, 9:S41-S53.
• [66]Williams I, McIver S, Moore D, Bryan S: The use of economic evaluations in NHS decision-making: a review and empirical investigation. Health Tech Assess 2008, 12(7):1-175. iii, ix-x
• [67]Colon-Ramos U, Lindsay A, Monge-Rojas R, Greaney M, Campos H, Peterson K: Translating research into action: a case study on trans fatty acid research and nutrition policy in Costa Rica. Health Pol Plann 2007, 22:363-374.
• [68]Hunsmann M: Limits to evidence-based health policymaking: policy hurdles to structural HIV prevention in Tanzania. Soc Sci Med 2012, 74:1477-1485.
• [69]Kok MO, Schuit AJ: Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Res Pol Syst 2012, 10:1-16.
• [70]Wehrens R, Bekker M, Bal R: The construction of evidence-based local health policy through partnerships: research infrastructure, process, and context in the Rotterdam ‘Healthy in the City’ programme. J Publ Health Pol 2010, 31:447-460.
• [71]Wehrens R, Bekker M, Bal R: Coordination of research, policy and practice: a case study of collaboration in the field of public health. Sci Public Policy 2011, 38:755-766.
• [72]Berridge V: Health and Society in Britain since 1939. 38th edition. Cambridge: Cambridge University Press; 1999.
• [73]Elliott H, Popay J: How are policy makers using evidence? Models of research utilisation and local NHS policy making. J Epidemiol Community Health 2000, 54:461-468.
• [74]McDonough JE: Experiencing Politics: A Legislator’s Stories of Government and Health Care. 2nd edition. Oakland, CA: University of California Press; 2000.
• [75]Haynes A, Derrick G, Redman S, Hall W, Gillespie J: Identifying trustworthy experts: how do policymakers find and assess public health researchers worth consulting or collaborating with? PLoS ONE 2012, 7:e32665.
• [76]Innvaer S: The use of evidence in public governmental reports on health policy: an analysis of 17 Norwegian official reports (NOU). BMC Health Serv Res 2009, 9:177.
• [77]Buse K, Mays N, Walt G: Making Health Policy. Maidenhead: Open University Press; 2005.
• [78]Oliver K, Kislov R: Policy, evidence and theory: contextualising a glossary of policymaking. J Epidemiol Community Health 2012. Response to A glossary of theories for understanding policymaking. Smith, Katikireddi 67:2 198-202 doi:10.1136/jech-2012-200990
• [79]Lasswell HD: Politics: Who Gets What, When, How. New York: P. Smith; 1950.
• [80]Gold M: Pathways to the use of health services research in policy. Health Serv Res 2009, 44:1111-1136.
• [81]Taylor-Robinson D, Milton B, Lloyd-Williams F, O'Flaherty M, Capewell S: Planning ahead in public health? A qualitative study of the time horizons used in public health decision-making. BMC Public Health 2008, 8:415.
• [82]Exworthy M, Hunter D: The Challenge of joined-up government in tackling health inequalities. Int J Publ Admin 2011, 34:201-212.
• [83]Sanderson I: Evaluation, policy learning and evidence-based policy making. Public Administration 2002, 80:1-22.
• [84]Dopson S, Locock L, Gabbay J, Ferlie E, FitzGerald L: Evidence-based medicine and the implementation gap. Health (London) 2003, 7:311-330.
• [85]Gorissen WHM, Schulpen TWJ, Kerkhoff AHM, van Heffen O: Bridging the gap between doctors and policymakers: The use of scientific knowledge in local school health care policy in the Netherlands. Eur J Public Health 2005, 15:133-139.
• [86]Armstrong R, Doyle J, Lamb C, Waters E: Multi-sectoral health promotion and public health: the role of evidence. J Public Health 2006, 28:168-172.
• [87]Choi BCK, Pang T, Lin V, Puska P, Sherman G, Goddard M, Ackland MJ, Sainsbury P, Stachenko S, Morrison H, Clottey C: Can scientists and policy makers work together? J Epidemiol Community Health 2005, 59:632-637.
• [88]Jasanoff S: States of Knowledge: The Co-Production of Science and the Social Order. London: Routledge; 2013.
• [89]Farrington : What works in preventing crime? Systematic reviews of experimental and quasi-experimental research. Ann Am Acad Pol Soc Sci 2001, 578:8.
• [90]Ritchie J, Spencer L: Qualitative data analysis for applied policy research. In Analyzing Qualitative Data. Edited by Bryman A, Burgess R. London: Routledge; 1999.
• [91]Petticrew M, Whitehead M, Macintyre SJ, Graham H, Egan M: Evidence for public health policy on inequalities: 1: The reality according to policymakers. J Epidemiol Community Health 2004, 58:811-816.
• [92]Ettelt S, Mays N: Health services research in Europe and its use for informing policy. J Health Serv Res Policy 2011, 16(Suppl 2):48-60.
• [93]Coleman PN: Influence of evidence-based guidance on health policy and clinical practice in England. Qual Health Care 2001, 10:229-237.
• [94]Ritter A: How do drug policy makers access research evidence? Int J Drug Pol 2009, 20:70-75.
• [95]Oliver K, de Vocht F: Defining ‘evidence’: a survey of public health policy makers’ needs and preferences. Eur J Public Health 2013. In press
• [96]Bullock H, Mountford J, Stanley R, Britain G: Better policy-making. London: Cabinet Office; 2001.
• [97]Boaz A, Grayson L, Levitt R, Solesbury W: Does evidence-based policy work? Learning from the UK experience. Evid Policy 2008, 4:233-253.
• [98]Cameron A, Lart R, Salisbury C, Purdy S, Thorp H, Stewart K, Peckham S, Calnan M, Purdy S, Thorp H: Policy makers’ perceptions on the use of evidence from evaluations. Evid Policy 2011, 7:429-448.
• [99]Macintyre S, Chalmers I, Horton R, Smith R: Using evidence to inform health policy: case study. BMJ 2001, 322:222-225.
• [100]Banks G: Evidence-Based Policy Making: What is it? How do we get it?. Canberra: Australian Government Productivity Commission; 2009.
• [101]Davies TO, Nutley S, Walter I: Approaches to Assessing Research Impact. 2005. Report of the ESRC Symposium on Assessing the Non-Academic Impact of Research. 2005. [http://www.esrc.ac.uk/_images/non-academic_impact_symposium_report_tcm8-3813.pdf webcite]
• [102]Penfield T, Baker MJ, Scoble R, Wykes MC: Assessment, evaluations, and definitions of research impact: A review. Res Eval 2014, 23:21-32.
• [103]Van Kammen J, de Savigny D, Sewankambo N: Using knowledge brokering to promote evidence-based policy-making: the need for support structures. Bull World Health Organ 2006, 84:608-612.
• [104]Coe R, Fitz-Gibbon C, Tymms P: Promoting Evidence-Based Education: The Role of Practitioners. Cardiff, Wales: Roundtable presentation to the British Educational Research Association’s Annual Conference; 2000.
• [105]Berridge V: Passive smoking and its pre-history in Britain: policy speaks to science? Soc Sci Med 1999, 49:1183-1195.
• [106]Bijker WE, Bal R, Hendriks R: The Paradox of Scientific Authority: The Role of Scientific Advice in Democracies. Cambridge, MA: MIT Press; 2009.
• [107]Cairney P: A ‘multiple lenses’ approach to policy change: the case of tobacco policy in the UK. British Politics 2007, 2:45-68.
• [108]Jasanoff S: The Fifth Branch: Science Advisers as Policymakers. Cambridge, MA: Harvard University Press; 1994.
• [109]Lipsky M: Street-Level Bureaucracy. New York: Russell Sage Foundation; 2010.
• [110]Checkland K, Snow S, McDermott I, Harrison S, Coleman A: Animateurs and animation: what makes a good commissioning manager? J Health Serv Res Policy 2012, 17:11-17.
• [111]Oliver K, de Vocht F, Money A, Everett MG: Who runs public health? A mixed-methods study combining network and qualitative analyses. J Public Health 2013, 35:453-459.
• [112]Head BW: Three lenses of evidence - based policy. Aust J Publ Admin 2008, 67:1-11.
• [113]Head BW: Reconsidering evidence-based policy: key issues and challenges. Pol Soc 2010, 29:77-94.
• [114]Cohen M, March J, Olsen J: A garbage can model of organisational choice. Admin Sci Q 1972, 17:1-25.
• [115]Fotaki M: Why do public policies fail so often? Exploring health policy-making as an imaginary and symbolic construction. Organization 2010, 17:703-720.
• [116]Yanow D: A Policy ethnographer’s reading of policy anthropology. In Policy Worlds: Anthropology and the Analysis of Contemporary Power. New York: Berghahn Books; 2011:300-313.
• [117]Yin RK: Case Study Research: Design and Methods. Thousand Oaks, CA: Sage Publications; 2014.
• [118]Bourke A: Universities, civil society and the global agenda of community-engaged research. Globalisation, Soc Educ 2013, 11:498-519.
• [119]Dobrow MJ, Goel V, Lemieux-Charles L, Black NA: The impact of context on evidence utilization: a framework for expert groups developing health policy recommendations. Soc Sci Med 2006, 63:1811-1824.
• [120]Wendt C: Mapping European healthcare systems: a comparative analysis of financing, service provision and access to healthcare. J Eur Soc Pol 2009, 19:432-445.