Learning attention for object tracking with adversarial learning network

EURASIP Journal on Image and Video Processing

Yongxiang Gu¹  Chen Song¹  Xu Cheng²  Beijing Chen² 

[1] School of Computer and Software, Nanjing University of Information Science and Technology, 210044, Nanjing, China;School of Computer and Software, Nanjing University of Information Science and Technology, 210044, Nanjing, China;Jiangsu Key Laboratory of Big Data Analysis Technology, Nanjing University of Information Science and Technology, 210044, Nanjing, China;Engineering Research Center of Digital Forensics, Ministry of Education, Nanjing University of Information Science and Technology, 210044, Nanjing, China;

关键词: Surveillance;    Deep learning;    Object tracking;    Generative adversarial learning;   

DOI  :  10.1186/s13640-020-00535-1

来源: Springer

【 摘 要 】

Artificial intelligence has been widely studied on solving intelligent surveillance analysis and security problems in recent years. Although many multimedia security approaches have been proposed by using deep learning network model, there are still some challenges on their performances which deserve in-depth research. On the one hand, high computational complexity of current deep learning methods makes it hard to be applied to real-time scenario. On the other hand, it is difficult to obtain the specific features of a video by fine-tuning the network online with the object state of the first frame, which fails to capture rich appearance variations of the object. To solve above two issues, in this paper, an effective object tracking method with learning attention is proposed to achieve the object localization and reduce the training time in adversarial learning framework. First, a prediction network is designed to track the object in video sequences. The object positions of the first ten frames are employed to fine-tune prediction network, which can fully mine a specific features of an object. Second, the prediction network is integrated into the generative adversarial network framework, which randomly generates masks to capture object appearance variations via adaptively dropout input features. Third, we present a spatial attention mechanism to improve the tracking performance. The proposed network can identify the mask that maintains the most robust features of the objects over a long temporal span. Extensive experiments on two large-scale benchmarks demonstrate that the proposed algorithm performs favorably against state-of-the-art methods.

【 授权许可】

CC BY