Deep Multi-Semantic Fusion-Based Cross-Modal Hashing

Mathematics

Liewu Cai¹  Xinghui Zhu¹  Zhuoyang Zou¹  Lei Zhu¹ 

[1] College of Information and Intelligence, Hunan Agricultural University, Changsha 410128, China;

关键词: cross-modal hashing;    semantic label information;    multi-label semantic fusion;    graph regularization;    deep neural network;   

DOI  :  10.3390/math10030430

来源: DOAJ

【 摘 要 】

Due to the low costs of its storage and search, the cross-modal retrieval hashing method has received much research interest in the big data era. Due to the application of deep learning, the cross-modal representation capabilities have risen markedly. However, the existing deep hashing methods cannot consider multi-label semantic learning and cross-modal similarity learning simultaneously. That means potential semantic correlations among multimedia data are not fully excavated from multi-category labels, which also affects the original similarity preserving of cross-modal hash codes. To this end, this paper proposes deep multi-semantic fusion-based cross-modal hashing (DMSFH), which uses two deep neural networks to extract cross-modal features, and uses a multi-label semantic fusion method to improve cross-modal consistent semantic discrimination learning. Moreover, a graph regularization method is combined with inter-modal and intra-modal pairwise loss to preserve the nearest neighbor relationship between data in Hamming subspace. Thus, DMSFH not only retains semantic similarity between multi-modal data, but integrates multi-label information into modal learning as well. Extensive experimental results on two commonly used benchmark datasets show that our DMSFH is competitive with the state-of-the-art methods.

【 授权许可】

Unknown