Deep learning with language models improves named entity recognition for PharmaCoNER

BMC Bioinformatics

Lei Wang¹  Yin Zhang¹  Zhihao Yang²  Hongfei Lin²  Jian Wang²  Cong Sun² 

[1] Beijing Institute of Health Administration and Medical Information, Beijing, China;School of Computer Science and Technology, Dalian University of Technology, Dalian, China;

关键词: Named entity recognition;    NER;    Language model;    BERT;    Text mining;   

DOI  :  10.1186/s12859-021-04260-y

来源: Springer

【 摘 要 】

BackgroundThe recognition of pharmacological substances, compounds and proteins is essential for biomedical relation extraction, knowledge graph construction, drug discovery, as well as medical question answering. Although considerable efforts have been made to recognize biomedical entities in English texts, to date, only few limited attempts were made to recognize them from biomedical texts in other languages. PharmaCoNER is a named entity recognition challenge to recognize pharmacological entities from Spanish texts. Because there are currently abundant resources in the field of natural language processing, how to leverage these resources to the PharmaCoNER challenge is a meaningful study.MethodsInspired by the success of deep learning with language models, we compare and explore various representative BERT models to promote the development of the PharmaCoNER task.ResultsThe experimental results show that deep learning with language models can effectively improve model performance on the PharmaCoNER dataset. Our method achieves state-of-the-art performance on the PharmaCoNER dataset, with a max F1-score of 92.01%.ConclusionFor the BERT models on the PharmaCoNER dataset, biomedical domain knowledge has a greater impact on model performance than the native language (i.e., Spanish). The BERT models can obtain competitive performance by using WordPiece to alleviate the out of vocabulary limitation. The performance on the BERT model can be further improved by constructing a specific vocabulary based on domain knowledge. Moreover, the character case also has a certain impact on model performance.

【 授权许可】

CC BY