Multi-modal entity alignment in hyperbolic space

NEUROCOMPUTING

卷:461

Article

Guo, Hao¹  Tang, Jiuyang¹  Zeng, Weixin¹  Zhao, Xiang¹  Liu, Li¹ 

[1] Natl Univ Def Technol, Changsha, Peoples R China

关键词: Multi-modal knowledge graphs;    Entity alignment;    Hyperbolic Graph Convolutional Networks;    Hyperboloid manifold;   

DOI  :  10.1016/j.neucom.2021.03.132

来源: Elsevier

【 摘 要 】

Many AI-related tasks involve the interactions of data in multiple modalities. It has been a new trend to merge multi-modal information into knowledge graph (KG), resulting in multi-modal knowledge graphs (MMKG). However, MMKGs usually suffer from low coverage and incompleteness. To mitigate this prob-lem, a viable approach is to integrate complementary knowledge from other MMKGs. To this end, although existing entity alignment approaches could be adopted, they operate in the Euclidean space, and the resulting Euclidean entity representations can lead to large distortion of KG's hierarchical struc-ture. Besides, the visual information has yet not been well exploited. In response to these issues, in this work, we propose a novel multi-modal entity alignment approach, Hyperbolic multi-modal entity alignment (HMEA), which extends the Euclidean representation to hyper-boloid manifold. We first adopt the Hyperbolic Graph Convolutional Networks (HGCNs) to learn struc-tural representations of entities. Regarding the visual information, we generate image embeddings using the densenet model, which are also projected into the hyperbolic space using HGCNs. Finally, we combine the structure and visual representations in the hyperbolic space and use the aggregated embed-dings to predict potential alignment results. Extensive experiments and ablation studies demonstrate the effectiveness of our proposed model and its components. CO 2021 Published by Elsevier B.V.

【 授权许可】

Free