Hierarchical shared transfer learning for biomedical named entity recognition

BMC Bioinformatics

Zhaoying Chai¹  Shenghui Shi¹  Han Jin¹  Yu Yang²  Lin Zhuo³  Siyan Zhan⁴ 

[1] College of Information Science and Technology, Beijing University of Chemical Technology, Beijing, China;National Institute of Health Data Science, Peking University, Beijing, China;Research Center of Clinical Epidemiology, Peking University Third Hospital, Beijing, China;School of Public Health, Peking University, Beijing, China;

关键词: BioNLP;    Biomedical named entity recognition;    Transfer learning;    Permutation language model;    Conditional random field;   

DOI  :  10.1186/s12859-021-04551-4

来源: Springer

【 摘 要 】

BackgroundBiomedical named entity recognition (BioNER) is a basic and important medical information extraction task to extract medical entities with special meaning from medical texts. In recent years, deep learning has become the main research direction of BioNER due to its excellent data-driven context coding ability. However, in BioNER task, deep learning has the problem of poor generalization and instability.Resultswe propose the hierarchical shared transfer learning, which combines multi-task learning and fine-tuning, and realizes the multi-level information fusion between the underlying entity features and the upper data features. We select 14 datasets containing 4 types of entities for training and evaluate the model. The experimental results showed that the F1-scores of the five gold standard datasets BC5CDR-chemical, BC5CDR-disease, BC2GM, BC4CHEMD, NCBI-disease and LINNAEUS were increased by 0.57, 0.90, 0.42, 0.77, 0.98 and − 2.16 compared to the single-task XLNet-CRF model. BC5CDR-chemical, BC5CDR-disease and BC4CHEMD achieved state-of-the-art results.The reasons why LINNAEUS’s multi-task results are lower than single-task results are discussed at the dataset level.ConclusionCompared with using multi-task learning and fine-tuning alone, the model has more accurate recognition ability of medical entities, and has higher generalization and stability.

【 授权许可】

CC BY