Multi-scale fusion visual attention network for facial micro-expression recognition

Frontiers in Neuroscience

Neuroscience

Hang Pan¹  Hongling Yang¹  Zhiliang Wang²  Lun Xie² 

[1] Department of Computer Science, Changzhi University, Changzhi, China;School of Computer and Communication Engineering, University of Science and Technology Beijing, Beijing, China;

关键词: micro-expression recognition;    attention mechanism;    mask operate;    multi-scale feature;    feature fusion;   

DOI  :  10.3389/fnins.2023.1216181

 received in 2023-05-03, accepted in 2023-06-26,  发布年份 2023

来源: Frontiers

【 摘 要 】

IntroductionMicro-expressions are facial muscle movements that hide genuine emotions. In response to the challenge of micro-expression low-intensity, recent studies have attempted to locate localized areas of facial muscle movement. However, this ignores the feature redundancy caused by the inaccurate locating of the regions of interest.MethodsThis paper proposes a novel multi-scale fusion visual attention network (MFVAN), which learns multi-scale local attention weights to mask regions of redundancy features. Specifically, this model extracts the multi-scale features of the apex frame in the micro-expression video clips by convolutional neural networks. The attention mechanism focuses on the weights of local region features in the multi-scale feature maps. Then, we mask operate redundancy regions in multi-scale features and fuse local features with high attention weights for micro-expression recognition. The self-supervision and transfer learning reduce the influence of individual identity attributes and increase the robustness of multi-scale feature maps. Finally, the multi-scale classification loss, mask loss, and removing individual identity attributes loss joint to optimize the model.ResultsThe proposed MFVAN method is evaluated on SMIC, CASME II, SAMM, and 3DB-Combined datasets that achieve state-of-the-art performance. The experimental results show that focusing on local at the multi-scale contributes to micro-expression recognition.DiscussionThis paper proposed MFVAN model is the first to combine image generation with visual attention mechanisms to solve the combination challenge problem of individual identity attribute interference and low-intensity facial muscle movements. Meanwhile, the MFVAN model reveal the impact of individual attributes on the localization of local ROIs. The experimental results show that a multi-scale fusion visual attention network contributes to micro-expression recognition.

【 授权许可】

Unknown   
Copyright © 2023 Pan, Yang, Xie and Wang.