Trajectory-Based Air-Writing Recognition Using Deep Neural Network and Depth Sensor

Sensors

Md.Ashraful Alam¹  Md.Shahinur Alam²  MohammedY. Abbass²  Nam Kim²  ShariarMd Imtiaz²  Ki-Chul Kwon² 

[1] Department of Computer Science and Engineering, BRAC University, Dhaka 1212, Bangladesh;Department of Computer and Communication Engineering, Chungbuk National University, Cheongju, Chungbuk 28644, Korea;

关键词: air-writing;    digit recognition;    depth sensor;    tof camera;    lstm;    cnn;    trajectory recognition;    human-computer interaction;   

DOI  :  10.3390/s20020376

来源: DOAJ

【 摘 要 】

Trajectory-based writing system refers to writing a linguistic character or word in free space by moving a finger, marker, or handheld device. It is widely applicable where traditional pen-up and pen-down writing systems are troublesome. Due to the simple writing style, it has a great advantage over the gesture-based system. However, it is a challenging task because of the non-uniform characters and different writing styles. In this research, we developed an air-writing recognition system using three-dimensional (3D) trajectories collected by a depth camera that tracks the fingertip. For better feature selection, the nearest neighbor and root point translation was used to normalize the trajectory. We employed the long short-term memory (LSTM) and a convolutional neural network (CNN) as a recognizer. The model was tested and verified by the self-collected dataset. To evaluate the robustness of our model, we also employed the 6D motion gesture (6DMG) alphanumeric character dataset and achieved 99.32% accuracy which is the highest to date. Hence, it verifies that the proposed model is invariant for digits and characters. Moreover, we publish a dataset containing 21,000 digits; which solves the lack of dataset in the current research.

【 授权许可】

Unknown