Classifying publications from the clinical and translational science award program along the translational research spectrum: a machine learning approach

Journal of Translational Medicine

Research

Emily Connors¹  Yindalon Aphinyanaphongs²  Trisha Adamus³  Alisa Surkis⁴  Joe D. Hunt⁵  Janice A. Hogle⁶  Meridith Mueller⁷  Elizabeth C. Whipple⁸  Paul E. Mazmanian⁹  Deborah DiazGranados⁹  Kate Westaby^1,10 

[1] Clinical and Translational Science Institute, Medical College of Wisconsin, Milwaukee, USA;Department of Population Health, NYU School of Medicine, New York, USA;Ebling Library for the Health Sciences, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, USA;Health Sciences Library, NYU School of Medicine, New York, USA;Indiana Clinical and Translational Sciences Institute, Indiana University School of Medicine, Indianapolis, USA;Institute for Clinical and Translational Research, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, USA;Population Health Sciences, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, USA;Ruth Lilly Medical Library, Indiana University School of Medicine, Indianapolis, USA;School of Medicine, Virginia Commonwealth University, Richmond, USA;Wisconsin Partnership Program, School of Medicine and Public Health, University of Wisconsin-Madison, Madison, USA;

关键词: Machine learning;    Translational research;    Knowledge translation;    Text classification;   

DOI  :  10.1186/s12967-016-0992-8

 received in 2016-06-01, accepted in 2016-07-27,  发布年份 2016

来源: Springer

【 摘 要 】

BackgroundTranslational research is a key area of focus of the National Institutes of Health (NIH), as demonstrated by the substantial investment in the Clinical and Translational Science Award (CTSA) program. The goal of the CTSA program is to accelerate the translation of discoveries from the bench to the bedside and into communities. Different classification systems have been used to capture the spectrum of basic to clinical to population health research, with substantial differences in the number of categories and their definitions. Evaluation of the effectiveness of the CTSA program and of translational research in general is hampered by the lack of rigor in these definitions and their application. This study adds rigor to the classification process by creating a checklist to evaluate publications across the translational spectrum and operationalizes these classifications by building machine learning-based text classifiers to categorize these publications.MethodsBased on collaboratively developed definitions, we created a detailed checklist for categories along the translational spectrum from T0 to T4. We applied the checklist to CTSA-linked publications to construct a set of coded publications for use in training machine learning-based text classifiers to classify publications within these categories. The training sets combined T1/T2 and T3/T4 categories due to low frequency of these publication types compared to the frequency of T0 publications. We then compared classifier performance across different algorithms and feature sets and applied the classifiers to all publications in PubMed indexed to CTSA grants. To validate the algorithm, we manually classified the articles with the top 100 scores from each classifier.ResultsThe definitions and checklist facilitated classification and resulted in good inter-rater reliability for coding publications for the training set. Very good performance was achieved for the classifiers as represented by the area under the receiver operating curves (AUC), with an AUC of 0.94 for the T0 classifier, 0.84 for T1/T2, and 0.92 for T3/T4.ConclusionsThe combination of definitions agreed upon by five CTSA hubs, a checklist that facilitates more uniform definition interpretation, and algorithms that perform well in classifying publications along the translational spectrum provide a basis for establishing and applying uniform definitions of translational research categories. The classification algorithms allow publication analyses that would not be feasible with manual classification, such as assessing the distribution and trends of publications across the CTSA network and comparing the categories of publications and their citations to assess knowledge transfer across the translational research spectrum.

【 授权许可】

CC BY   
© The Author(s) 2016