Research impact in the community-based health sciences: an analysis of 162 case studies from the 2014 UK Research Excellence Framework

BMC Medicine

Nick Fahy¹  Trisha Greenhalgh¹ 

[1]Nuffield Department of Primary Care Health Sciences, University of Oxford, New Radcliffe House, Radcliffe Observatory Quarter, Woodstock Road, Oxford OX2 6GG, UK

关键词: Knowledge translation;    Research impact;    Public health;    Primary care;   

Others  :  1226298
DOI  :  10.1186/s12916-015-0467-4

 received in 2015-08-06, accepted in 2015-08-28,  发布年份 2015

【 摘 要 】

Background

The 2014 UK Research Excellence Framework (REF2014) generated a unique database of impact case studies, each describing a body of research and impact beyond academia. We sought to explore the nature and mechanism of impact in a sample of these.

Methods

The study design was manual content analysis of a large sample of impact case studies (producing mainly quantitative data), plus in-depth interpretive analysis of a smaller sub-sample (for qualitative detail), thereby generating both breadth and depth. For all 162 impact case studies submitted to sub-panel A2 in REF2014, we extracted data on study design(s), stated impacts and audiences, mechanisms of impact, and efforts to achieve impact. We analysed four case studies (selected as exemplars of the range of approaches to impact) in depth, including contacting the authors for their narratives of impact efforts.

Results

Most impact case studies described quantitative research (most commonly, trials) and depicted a direct, linear link between research and impact. Research was said to have influenced a guideline in 122 case studies, changed policy in 88, changed practice in 84, improved morbidity in 44 and reduced mortality in 25. Qualitative and participatory research designs were rare, and only one case study described a co-production model of impact. Eighty-two case studies described strong and ongoing linkages with policymakers, but only 38 described targeted knowledge translation activities. In 40 case studies, no active efforts to achieve impact were described. Models of good implementation practice were characterised by an ethical commitment by researchers, strong institutional support and a proactive, interdisciplinary approach to impact activities.

Conclusion

REF2014 both inspired and documented significant efforts by UK researchers to achieve impact. But in contrast with the published evidence on research impact (which depicts much as occurring indirectly through non-linear mechanisms), this sub-panel seems to have captured mainly direct and relatively short-term impacts one step removed from patient outcomes. Limited impacts on morbidity and mortality, and researchers’ relatively low emphasis on the processes and interactions through which indirect impacts may occur, are concerns. These findings have implications for multi-stakeholder research collaborations such as UK National Institute for Health Research Collaborations for Leadership in Applied Health Research and Care, which are built on non-linear models of impact.

【 授权许可】

   
2015 Greenhalgh and Fahy.

【 图 表 】

Fig. 1.

Fig. 2.

Fig. 3.

Fig. 4.

【 参考文献 】

• [1]Higher Education Funding Council for England. 2014 REF: Assessment framework and guidance on submissions. Panel A criteria. London (REF 01/2012): HEFCE; 2012.
• [2]Walshe K, Davies HT. Health research, development and innovation in England from 1988 to 2013: from research production to knowledge mobilization. J Health Serv Res Policy. 2013; 18:1-12.
• [3]Glasziou P, Altman DG, Bossuyt P, Boutron I, Clarke M, Julious S et al.. Reducing waste from incomplete or unusable reports of biomedical research. The Lancet. 2014; 383:267-276.
• [4]Raftery J, Hanney S, Greenhalgh T, Glover M, Young A. A systematic review of impact measures in health technology assessment. Under Review for Health Technology Assessment. 2015.
• [5]Penfield T, Baker MJ, Scoble R, Wykes MC. Assessment, evaluations, and definitions of research impact: a review. Research Evaluation. 2013. rvt021. doi:. 10. 1093/reseval/rvt021 webcite
• [6]Milat AJ, Bauman AE, Redman S. A narrative review of research impact assessment models and methods. Health Res Policy Syst. 2015; 13:18. BioMed Central Full Text
• [7]Grant J, Brutscher P-B, Kirk SE, Butler L, Wooding S. Capturing research impacts: a review of international practice. Documented briefing. Rand Corporation. 2010. http://www.rand.org/content/dam/rand/pubs/documented_briefings/2010/RAND_DB578.pdf. Accessed 16 September 2015.
• [8]Meagher L, Lyall C, Nutley S. Flows of knowledge, expertise and influence: a method for assessing policy and practice impacts from social science research. Res Eval. 2008; 17:163-173.
• [9]Weiss CH. The many meanings of research utilization. Public Adm Rev. 1979;426–431.
• [10]Gabbay J, May A. Evidence based guidelines or collectively constructed “mindlines?” Ethnographic study of knowledge management in primary care. BMJ. 2004; 329:1013.
• [11]Davies H, Nutley S, Walter I. Why ‘knowledge transfer’ is misconceived for applied social research. J Health Serv Res Policy. 2008; 13:188-190.
• [12]Contandriopoulos D, Lemire M, Denis J-L, Tremblay É. Knowledge exchange processes in organizations and policy arenas: a narrative systematic review of the literature. Milbank Quarterly. 2010; 88:444-483.
• [13]Grol R, Grimshaw J. From best evidence to best practice: effective implementation of change in patients’ care. The Lancet. 2003; 362:1225-1230.
• [14]Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. Diffusion of innovations in service organizations: systematic review and recommendations. Milbank Quarterly. 2004; 82:581-629.
• [15]Hanney SR, Gonzalez-Block MA, Buxton MJ, Kogan M. The utilisation of health research in policy-making: concepts, examples and methods of assessment. Health Res Policy Syst. 2003; 1:2. BioMed Central Full Text
• [16]Lomas J. The in-between world of knowledge brokering. BMJ. 2007; 334:129-132.
• [17]Lindblom CE. The science of“ muddling through”. Public Adm Rev. 1959;79–88.
• [18]Hughes A, Martin B. Enhancing impact: the value of public sector R&D. CIHE & UKirc, 2012. http://www.ncub.co.uk/impact. Accessed 16 September 2015.
• [19]Thorp H, Goldstein B. Engines of innovation: the entrepreneurial university in the twenty-first century. Chapel Hill. University of North Carolina Press Books, Chapel Hill, North Carolina; 2013.
• [20]Ramaswamy V, Gouillart F. Building the co-creative enterprise. Harv Bus Rev. 2010; 88:100-109.
• [21]Gibbons M, Limoges C, Nowotny H, Schwartzman S, Scott P, Trow M. The new production of knowledge: the dynamics of science and research in contemporary societies. Sage, London; 1994.
• [22]Kislov R, Waterman H, Harvey G, Boaden R. Rethinking capacity building for knowledge mobilisation: developing multilevel capabilities in healthcare organisations. Implement Sci. 2014; 9:166. BioMed Central Full Text
• [23]Buxton M, Hanney S. How can payback from health services research be assessed? J Health Serv Res Policy. 1996; 1:35-43.
• [24]Kok MO, Schuit AJ. Contribution mapping: a method for mapping the contribution of research to enhance its impact. Health Res Policy Syst. 2012; 10:21. BioMed Central Full Text
• [25]Long JC, Cunningham FC, Braithwaite J. Bridges, brokers and boundary spanners in collaborative networks: a systematic review. BMC Health Serv Res. 2013; 13:158. BioMed Central Full Text
• [26]Upton S, Vallance P, Goddard J. From outcomes to process: evidence for a new approach to research impact assessment. Res Eval. 2014; 23:352-365.
• [27]Spaapen J, van Drooge L. Introducing ‘productive interactions’ in social impact assessment. Res Eval. 2011; 20:211-218.
• [28]Hinrichs S, Grant J. A new resource for identifying and assessing the impacts of research. BMC Med. 2015; 13:148. BioMed Central Full Text
• [29]REF 2014 Results and Submissions. http://results.ref.ac.uk. Accessed 16 September 2015.
• [30]Jagosh J, Macaulay AC, Pluye P, Salsberg J, Bush PL, Henderson J et al.. Uncovering the benefits of participatory research: implications of a realist review for health research and practice. Milbank Quarterly. 2012; 90:311-346.
• [31]Fitzgerald L, Harvey G. Translational networks in healthcare? Evidence on the design and initiation of organizational networks for knowledge mobilization. Soc Sci Med. 2015; 138:192-200.
• [32]Higher Education Funding Council. Research Excellence Framework 2014: overview report by Main Panel A and sub-panels 1 to 6. London: HEFCE; 2015. Panel A overview report.pdf. http://www.ref.ac.uk/panels/paneloverviewreports/. Accessed 16 September 2015.
• [33]The Equality Trust. https://www.equalitytrust.org.uk. Accessed 16 September 2015.
• [34]The Equality Trust: Education. https://www.equalitytrustr.org.uk/education. Accessed 16 September 2015.
• [35]South J, Cattan M. Developing evidence for public health policy and practice: the implementation of a knowledge translation approach in a staged, multi-methods study in England, 2007–09. Evidence & Policy: A Journal of Research, Debate and Practice. 2014; 10:379-396.
• [36]Changing clinical practice in Bell’s Palsy: the impact of a clinical trial highlighting the impact of evidence for primary care. http://results.ref.ac.uk/DownloadFile/ImpactCaseStudy/pdf?caseStudyId=39367. Accessed 16 September 2015.
• [37]Bristol research leads to a worldwide fall in the number of cot deaths. http://results.ref.ac.uk/DownloadFile/ImpactCaseStudy/pdf?caseStudyId=40166. Accessed 16 September 2015.
• [38]The Impact of Social Inequality: changing the public and policy debate. http://results.ref.ac.uk/DownloadFile/ImpactCaseStudy/pdf?caseStudyId=43415. Accessed 16 September 2015.
• [39]Developing evidence based practice on lay health roles. http://results.ref.ac.uk/DownloadFile/ImpactCaseStudy/pdf?caseStudyId=13837. Accessed 16 September 2015.