Identifying potential indicators to measure the outcome of translational cancer research: a mixed methods approach

Health Research Policy and Systems

Corinne Alberti³  Mahasti Saghatchian²  Serge Gottot³  Maria Teixeira¹  Rym Boulkedid³  Frédérique Thonon³ 

[1] AP-HP, Hôpital Robert Debré, Unité d’épidémiologie clinique, Paris, France;European and International Affairs Unit, Gustave Roussy, Villejuif, France;INSERM, U 1123 and CIC-EC 1426, ECEVE, Paris, France

关键词: Translational research;    Research impact;    Qualitative research;    Indicators;    Delphi technique;    Cancer;   

Others  :  1235211
DOI  :  10.1186/s12961-015-0060-5

 received in 2015-07-13, accepted in 2015-11-12,  发布年份 2015

【 摘 要 】

Background

In a context where there is an increasing demand to evaluate the outcome of bio-medical research, our work aims to develop a set of indicators to measure the impact of translational cancer research. The objective of our study was to explore the scope and issues of translational research relevant to evaluation, explore the views of researchers on the evaluation of oncological translational research, and select indicators measuring the outcomes and outputs of translational research in oncology by consensus.

Methods

Semi-structured interviews amongst 23 researchers involved in translational cancer research were conducted and analysed using thematic analysis. A two-round modified Delphi survey of 35 participants with similar characteristics was then performed followed by a physical meeting. Participants rated the feasibility and validity of 60 indicators. The physical meeting was held to discuss the methodology of the new indicators.

Results

The main themes emerging from the interviews included a common definition for translational research but disagreements about the exact scope and limits of this research, the importance of multidisciplinarity and collaboration for the success of translational research, the disadvantages that translational research faces in current evaluation systems, the relative lack of pertinence of existing indicators, and propositions to measure translational cancer research in terms of clinical applications and patient outcomes. A total of 35 participants took part in the first round survey and 12 in the second round. The two-round survey helped us select a set of 18 indicators, including four that seemed to be particularly adapted to measure translational cancer research impact on health service research (number of biomarkers identified, generation of clinical guidelines, citation of research in clinical guidelines, and citation of research in public health guidelines). The feedback from participants helped refine the methodology and definition of indicators not commonly used.

Conclusion

Indicators need to be accepted by stakeholders under evaluation. This study helped the selection and refinement of indicators considered as the most relevant by researchers in translational cancer research. The feasibility and validity of those indicators will be tested in a scientometric study.

【 授权许可】

   
2015 Thonon et al.

【 图 表 】

Figure 1.

【 参考文献 】

• [1]Butler D. Translational research: crossing the valley of death. Nature. 2008; 453(7197):840-842.
• [2]National Cancer Institute. Report of the Translational Research Working Group of the National Cancer Advisory Board. Transforming Translation – Harnessing Discovery for Patient and Public Benefit. 2007. http://www.cancer.gov/about-nci/organization/ccct/about/trwg-report.pdf. Accessed 1 September 2014
• [3]Trochim W, Kane C, Graham MJ, Pincus HA. Evaluating translational research: a process marker model. Clin Transl Sci. 2011; 4(3):153-162.
• [4]Wells R, Whitworth JA. Assessing outcomes of health and medical research: do we measure what counts or count what we can measure? Aust N Z Health Policy. 2007; 4:14. BioMed Central Full Text
• [5]Donabedian A. Evaluating the quality of medical care. Milbank Mem Fund Q. 1966; 44(3):Suppl:166-Suppl:206.
• [6]Pozen R, Kline H. Defining success for translational research organizations. Sci Transl Med. 2011; 3(94):94cm20.
• [7]Rajan A, Caldas C, van Luenen H, Saghatchian M, van Harten WH. Assessing excellence in translational cancer research: a consensus based framework. J Transl Med. 2013; 11:274. BioMed Central Full Text
• [8]Wellcome Trust, Academy of Medical Sciences, Medical Research Council. Medical research: assessing the benefits to society – A report by the UK Evaluation Forum, supported by the Academy of Medical Sciences, Medical Research Council and Wellcome Trust. Academy of Health Sciences, Medical Research Council, Wellcome Trust; 2006. http://www.acmedsci.ac.uk/policy/policy-projects/medical-research-assessing-the-benefits-to-society/. Accessed 20 January 2014.
• [9]Canadian Institutes of Health Research. Developing a CIHR framework to measure the impact of health research. 2005. http://publications.gc.ca/collections/Collection/MR21-65-2005E.pdf. Accessed 20 January 2014.
• [10]Thonon F, Boulkedid R, Delory T, Rousseau S, Saghatchian M, van Harten W et al.. Measuring the Outcome of Biomedical Research: A Systematic Literature Review. PLoS ONE. 2015; 10(4): Article ID e0122239
• [11]Kelley E, Hurst J. Health care quality indicators project- conceptual framework paper. OECD Publishing; 2006. http://www.oecd-ilibrary.org/docserver/download/5l9t19m240hc.pdf?expires=1445502732&id=id&accname=guest&checksum=09A27B6D33CC9E16D505C0C95826C08F. Accessed 20 January 2014.
• [12]Greene JC, Caracelli VJ, Graham WF. Toward a conceptual framework for mixed-method evaluation designs. Educ Eval Policy Anal. 1989; 11(3):255-274.
• [13]Boulkedid R, Abdoul H, Loustau M, Sibony O, Alberti C. Using and reporting the Delphi method for selecting healthcare quality indicators: a systematic review. PLoS ONE. 2011; 6(6):e20476.
• [14]Kuper A, Lingard L, Levinson W. Critically appraising qualitative research. BMJ. 2008; 337:a1035.
• [15]Burnard P. A method of analysing interview transcripts in qualitative research. Nurse Educ Today. 1991; 11(6):461-466.
• [16]Tong A, Sainsbury P, Craig J. Consolidated criteria for reporting qualitative research (COREQ): a 32-item checklist for interviews and focus groups. Int J Qual Health Care J Int Soc Qual Health Care ISQua. 2007; 19(6):349-357.
• [17]Dougherty D, Conway PH. The “3T’s” road map to transform US health care: the “how” of high-quality care. JAMA. 2008; 299(19):2319-2321.
• [18]Woolf SH. The meaning of translational research and why it matters. JAMA. 2008; 299(2):211-213.
• [19]Henry NL, Hayes DF. Cancer biomarkers. Mol Oncol. 2012; 6(2):140-146.
• [20]Parkinson DR, Johnson BE, Sledge GW. Making personalized cancer medicine a reality: challenges and opportunities in the development of biomarkers and companion diagnostics. Clin Cancer Res Off J Am Assoc Cancer Res. 2012; 18(3):619-624.
• [21]Bast RC, Lilja H, Urban N, Rimm DL, Fritsche H, Gray J et al.. Translational crossroads for biomarkers. Clin Cancer Res Off J Am Assoc Cancer Res. 2005; 11(17):6103-6108.
• [22]Tan DSW, Thomas GV, Garrett MD, Banerji U, de Bono JS, Kaye SB et al.. Biomarker-driven early clinical trials in oncology: a paradigm shift in drug development. Cancer J Sudbury Mass. 2009; 15(5):406-420.
• [23]Onwuegbuzie A, Leech N. On becoming a pragmatic researcher: the importance of combining quantitative and qualitative research methodologies. Int J Soc Res Methodol Theory Pract. 2005; 8(5):375-387.
• [24]Polit DF, Beck CT. Generalization in quantitative and qualitative research: myths and strategies. Int J Nurs Stud. 2010; 47(11):1451-1458.
• [25]Littman BH, Di Mario L, Plebani M, Marincola FM. What’s next in translational medicine? Clin Sci Lond Engl. 1979. 2007; 112(4):217-227.
• [26]Lord CJ, Ashworth A. Biology-driven cancer drug development: back to the future. BMC Biol. 2010; 8:38. BioMed Central Full Text
• [27]De Bono JS, Ashworth A. Translating cancer research into targeted therapeutics. Nature. 2010; 467(7315):543-549.
• [28]Pober JS, Neuhauser CS, Pober JM. Obstacles facing translational research in academic medical centers. FASEB J Off Publ Fed Am Soc Exp Biol. 2001; 15(13):2303-2313.
• [29]Zhang Y, Diao T, Wang L. Quantitative evaluation of translational medicine based on scientometric analysis and information extraction. Clin Transl Sci. 2014; 7(6):465-469.
• [30]Luukkonen T. Bibliometrics and evaluation of research performance. Ann Med. 1990; 22(3):145-150.