Multi-Term Attention Networks for Skeleton-Based Action Recognition

Applied Sciences

Xiaolei Diao¹  Chen Huang¹  Xiaoqiang Li¹ 

[1] School of Computer Engineering and Science, Shanghai University, Shanghai 200444, China;

关键词: action recognition;    skeleton sequences;    attention mechanism;    spatio-temporal features;   

DOI  :  10.3390/app10155326

来源: DOAJ

【 摘 要 】

The same action takes different time in different cases. This difference will affect the accuracy of action recognition to a certain extent. We propose an end-to-end deep neural network called “Multi-Term Attention Networks” (MTANs), which solves the above problem by extracting temporal features with different time scales. The network consists of a Multi-Term Attention Recurrent Neural Network (MTA-RNN) and a Spatio-Temporal Convolutional Neural Network (ST-CNN). In MTA-RNN, a method for fusing multi-term temporal features are proposed to extract the temporal dependence of different time scales, and the weighted fusion temporal feature is recalibrated by the attention mechanism. Ablation research proves that this network has powerful spatio-temporal dynamic modeling capabilities for actions with different time scales. We perform extensive experiments on four challenging benchmark datasets, including the NTU RGB+D dataset, UT-Kinect dataset, Northwestern-UCLA dataset, and UWA3DII dataset. Our method achieves better results than the state-of-the-art benchmarks, which demonstrates the effectiveness of MTANs.

【 授权许可】

Unknown