Hierarchical Graph Transformer-Based Deep Learning Model for Large-Scale Multi-Label Text Classification

IEEE Access

Mingsheng Liu¹  Md Zakirul Alam Bhuiyan²  Hongyuan Ma³  Qi Teng⁴  Linfeng Du⁴  Hekai Zhang⁵  Shuai Chen⁵  Zhiyong Teng⁵  Jibing Gong⁵  Jianhua Li⁶ 

[1] College of Electrical Engineering, Hebei University of Technology, Tianjin, China;Department of Computer and Information Sciences, Fordham University, New York, NY, USA;National Computer Network Emergency Response Technical Team/Coordination Center of China, Beijing, China;School of Computer Science and Engineering, Beihang University, Beijing, China;School of Information Science and Engineering, Yanshan University, Qinhuangdao, China;School of Information Science and Technology, Shijiazhuang Tiedao University, Shijiazhuang, China;

关键词: Multi-label text classification;    graph modeling;    graph transformer;    deep learning;   

DOI  :  10.1109/ACCESS.2020.2972751

来源: DOAJ

【 摘 要 】

Traditional methods of multi-label text classification, particularly deep learning, have achieved remarkable results. However, most of these methods use word2vec technology to represent sequential text information, while ignoring the logic and internal hierarchy of the text itself. Although these approaches can learn the hypothetical hierarchy and logic of the text, it is unexplained. In addition, the traditional approach treats labels as independent individuals and ignores the relationships between them, which not only does not reflect reality but also causes significant loss of semantic information. In this paper, we propose a novel Hierarchical Graph Transformer based deep learning model for large-scale multi-label text classification. We first model the text into a graph structure that can embody the different semantics of the text and the connections between them. We then use a multi-layer transformer structure with a multi-head attention mechanism at the word, sentence, and graph levels to fully capture the features of the text and observe the importance of the separate parts. Finally, we use the hierarchical relationship of the labels to generate the representation of the labels, and design a weighted loss function based on the semantic distances of the labels. Extensive experiments conducted on three benchmark datasets demonstrated that the proposed model can realistically capture the hierarchy and logic of text and improve performance compared with the state-of-the-art methods.

【 授权许可】

Unknown