Micro-expression recognition with small sample size by transferring long-term convolutional neural network

NEUROCOMPUTING

卷:312

Article

Wang, Su-Jing^1,8  Li, Bing-Jun²  Liu, Yong-Jin²  Yan, Wen-Jing³  Ou, Xinyu⁴  Huang, Xiaohua⁵  Xu, Feng⁶  Fu, Xiaolan^7,8 

[1] Inst Psychol, CAS Key Lab Behav Sci, Beijing 100101, Peoples R China

[2] Tsinghua Univ, Dept Comp Sci & Technol, Beijing Natl Res Ctr Informat Sci & Technol, Beijing, Peoples R China

[3] Northeastern Univ, Coll Informat Sci & Engn, Shenyang, Liaoning, Peoples R China

[4] Yunnan Open Univ, Cadres Online Learning Inst Yunnan Prov, Kunming 650223, Yunnan, Peoples R China

[5] Univ Oulu, Faulty Informat Technol & Elect Engn, Ctr Machine Vis & Signal Anal, POB 4500, FI-90014 Oulu, Finland

[6] Fudan Univ, Sch Comp Sci, Key Lab Informat Sci Electromagnet Waves MoE, Shanghai Key Lab Intelligent Informat Proc, Shanghai 200433, Peoples R China

[7] Chinese Acad Sci, Inst Psychol, State Key Lab Brain & Cognit Sci, Beijing 100101, Peoples R China

[8] Univ Chinese Acad Sci, Dept Psychol, Beijing 100049, Peoples R China

关键词: Micro-expression;    Deep learning;    Transferring learning;    Convolutional neural network;   

DOI  :  10.1016/j.neucom.2018.05.107

来源: Elsevier

【 摘 要 】

Micro-expression is one of important clues for detecting lies. Its most outstanding characteristics include short duration and low intensity of movement. Therefore, video clips of high spatial-temporal resolution are much more desired than still images to provide sufficient details. On the other hand, owing to the difficulties to collect and encode micro-expression data, it is small sample size. In this paper, we use only 560 micro-expression video clips to evaluate the proposed network model: Transferring Long-term Convolutional Neural Network (TLCNN). TLCNN uses Deep CNN to extract features from each frame of micro-expression video clips, then feeds them to Long Short Term Memory (LSTM) which learn the temporal sequence information of micro-expression. Due to the small sample size of micro-expression data, TLCNN uses two steps of transfer learning: (1) transferring from expression data and (2) transferring from single frame of micro-expression video clips, which can be regarded as big data. Evaluation on 560 micro-expression video clips collected from three spontaneous databases is performed. The results show that the proposed TLCNN is better than some state-of-the-art algorithms. (C) 2018 Elsevier B.V. All rights reserved.

【 授权许可】

Free