No Interface, No Problem: Gesture Recognition on Physical Objects Using Radar Sensing

Sensors

Luis A. Leiva¹  Alexander Plopski²  Klen Čopič Pucihar³  Matjaž Kljun³  Nuwan T. Attygalle³  Hirokazu Kato⁴  Christian Sandor⁵ 

[1] Department of Computer Science, University of Luxembourg, Maison du Nombre 6, Avenue de la Fonte, L-4364 Esch-sur-Alzette, Luxembourg;Department of Information Science, University of Otago, P.O. Box 56, Dunedin 9054, New Zealand;Faculty of Mathematics, Natural Sciences and Information Technologies (FAMNIT), University of Primorska, Glagoljaška 8, 6000 Koper, Slovenia;Graduate School of Science and Technology, Nara Institute of Science and Technology, Takayama 8916-5, Ikoma, Nara, Japan;School of Creative Media, City University of Hong Kong, Hong Kong, China;

关键词: radar sensing;    gesture recognition;    deep learning;    human factors;   

DOI  :  10.3390/s21175771

来源: DOAJ

【 摘 要 】

Physical objects are usually not designed with interaction capabilities to control digital content. Nevertheless, they provide an untapped source for interactions since every object could be used to control our digital lives. We call this the missing interface problem: Instead of embedding computational capacity into objects, we can simply detect users’ gestures on them. However, gesture detection on such unmodified objects has to date been limited in the spatial resolution and detection fidelity. To address this gap, we conducted research on micro-gesture detection on physical objects based on Google Soli’s radar sensor. We introduced two novel deep learning architectures to process range Doppler images, namely a three-dimensional convolutional neural network (Conv3D) and a spectrogram-based ConvNet. The results show that our architectures enable robust on-object gesture detection, achieving an accuracy of approximately 94% for a five-gesture set, surpassing previous state-of-the-art performance results by up to 39%. We also showed that the decibel (dB) Doppler range setting has a significant effect on system performance, as accuracy can vary up to 20% across the dB range. As a result, we provide guidelines on how to best calibrate the radar sensor.

【 授权许可】

Unknown