RGB-D static gesture recognition based on convolutional neural network

The Journal of Engineering

Bin Xie¹  Yi Li²  Xiaoyu He³ 

[1] Mobile Health Ministry of Education, China Mobile Joint Laboratory, Xiangya Hospital Central South University , Changsha , People'School of Information Science and Engineering, Central South University , Changsha , People's Republic of China

关键词: RGB-D camera;    traditional machine learning methods;    depth images;    RGB-D static gesture recognition method;    fine-tuning Inception V3;    robust gesture recognition system;    RGB input;    computer vision;    feature extraction;    ASL recognition dataset;    depth information;    gesture segmentation;    American Sign Language Recognition dataset;    depth camera;    CNN algorithms;    RGB input only method;    CNN structure;    convolutional neural network;   

DOI  :  10.1049/joe.2018.8327

学科分类：工程和技术（综合）

来源: IET

【 摘 要 】

In the area of human–computer interaction (HCI) and computer vision, gesture recognition has always been a research hotspot. With the appearance of depth camera, gesture recognition using RGB-D camera has gradually become mainstream in this field. However, how to effectively use depth information to construct a robust gesture recognition system is still a problem. In this paper, an RGB-D static gesture recognition method based on fine-tuning Inception V3 is proposed, which can eliminate the steps of gesture segmentation and feature extraction in traditional algorithms. Compared with general CNN algorithms, the authors adopt a two-stage training strategy to fine-tune the model. This method sets a feature concatenate layer of RGB and depth images in the CNN structure, using depth information to promote the performance of gesture recognition. Finally, on the American Sign Language (ASL) Recognition dataset, the authors compared their method with other traditional machine learning methods, CNN algorithms, and the RGB input only method. Among three groups of comparative experiments, the authors’ method reached the highest accuracy of 91.35%, reaching the state-of-the-art currently on ASL dataset.

【 授权许可】

CC BY