TDCMR: Triplet-Based Deep Cross-Modal Retrieval for Geo-Multimedia Data

Applied Sciences

Yunwu Lin¹  Jiayu Song¹  Leyuan Zhang²  Jiagang Song²  Weiren Yu³ 

[1] School of Computer Science and Engineering, Central South University, Changsha 410083, China;School of Computer Science and Engineering, Guangxi Normal University, Guiling 541004, China;School of Computer Science and Technology, Nanjing University of Science and Technology, Nanjing 210094, China;

关键词: geo-multimedia;    nearest neighbor query;    cross-modal hashing;    triplet loss;    TH-Quadtree;   

DOI  :  10.3390/app112210803

来源: DOAJ

【 摘 要 】

Mass multimedia data with geographical information (geo-multimedia) are collected and stored on the Internet due to the wide application of location-based services (LBS). How to find the high-level semantic relationship between geo-multimedia data and construct efficient index is crucial for large-scale geo-multimedia retrieval. To combat this challenge, the paper proposes a deep cross-modal hashing framework for geo-multimedia retrieval, termed as Triplet-based Deep Cross-Modal Retrieval (TDCMR), which utilizes deep neural network and an enhanced triplet constraint to capture high-level semantics. Besides, a novel hybrid index, called TH-Quadtree, is developed by combining cross-modal binary hash codes and quadtree to support high-performance search. Extensive experiments are conducted on three common used benchmarks, and the results show the superior performance of the proposed method.

【 授权许可】

Unknown