Improved itracker combined with bidirectional long short-term memory for 3D gaze estimation using appearance cues

NEUROCOMPUTING

卷:390

Article

Zhou, Xiaolong^1,2  Lin, Jianing¹  Zhang, Zhuo¹  Shao, Zhanpeng¹  Chen, Shenyong^1,3  Liu, Honghai⁴ 

[1] Zhejiang Univ Technol, Hangzhou 210023, Peoples R China

[2] Quzhou Univ, Quzhou 324000, Peoples R China

[3] Tianjin Univ Technol, Tianjin 300384, Peoples R China

[4] Univ Portsmouth, Portsmouth, Hants, England

关键词: Gaze estimation;    CNN;    RNN;    LSTM;   

DOI  :  10.1016/j.neucom.2019.04.099

来源: Elsevier

【 摘 要 】

Gaze is an important non-verbal cue for speculating human's attention, which has been widely employed in many human-computer interaction-based applications. In this paper, we propose an improved Itracker to predict the subject's gaze for a single image frame, as well as employ a many-to-one bidirectional Long Short-Term Memory (bi-LSTM) to fit the temporal information between frames to estimate gaze for video sequence. For single image frame gaze estimation, we improve the conventional Itracker by removing the face-grid and reducing one network branch via concatenating the two-eye region images. Experimental results show that our improved Itracker obtains 11.6% significant improvement over the state-of-the-art methods on MPIIGaze dataset and has robust estimation accuracy for different image resolutions under the premise of greatly reducing network complexity. For video sequence gaze estimation, by employing the bi-LSTM to fit the temporal information between frames, experimental results on EyeDiap dataset further demonstrate 3% accuracy improvement. (C) 2019 Published by Elsevier B.V.

【 授权许可】

Free