Distributed representation and one-hot representation fusion with gated network for clinical semantic textual similarity

BMC Medical Informatics and Decision Making

Haoming Qin¹  Yedan Shen¹  Xiaolong Wang¹  He Cao¹  Shuai Chen¹  Qingcai Chen¹  Buzhou Tang¹  Ying Xiong¹  Jun Yan² 

[1] Department of Computer Science, Harbin Institute of Technology;Yidu Cloud (Beijing) Technology Co., Ltd;

关键词: Clinical semantic textual similarity;    Gated network;    Distributed representation;    One-hot representation;   

DOI  :  10.1186/s12911-020-1045-z

来源: DOAJ

【 摘 要 】

Abstract Background Semantic textual similarity (STS) is a fundamental natural language processing (NLP) task which can be widely used in many NLP applications such as Question Answer (QA), Information Retrieval (IR), etc. It is a typical regression problem, and almost all STS systems either use distributed representation or one-hot representation to model sentence pairs. Methods In this paper, we proposed a novel framework based on a gated network to fuse distributed representation and one-hot representation of sentence pairs. Some current state-of-the-art distributed representation methods, including Convolutional Neural Network (CNN), Bi-directional Long Short Term Memory networks (Bi-LSTM) and Bidirectional Encoder Representations from Transformers (BERT), were used in our framework, and a system based on this framework was developed for a shared task regarding clinical STS organized by BioCreative/OHNLP in 2018. Results Compared with the systems only using distributed representation or one-hot representation, our method achieved much higher Pearson correlation. Among all distributed representations, BERT performed best. The highest Person correlation of our system was 0.8541, higher than the best official one of the BioCreative/OHNLP clinical STS shared task in 2018 (0.8328) by 0.0213. Conclusions Distributed representation and one-hot representation are complementary to each other and can be fused by gated network.

【 授权许可】

Unknown