Towards hand gesture-based control of virtual keyboards for effective communication

Conference on Innovation in Technology and Engineering Science

工业技术（总论）

Rusydi, Muhammad Ilhamdi^1 ; Oktrison^1 ; Azhar, Willy^1 ; Oluwarotimi, Samuel W^2 ; Rusydi, Febdian^3

Department of Electrical Engineering, Universitas Andalas, Padang, Indonesia^1

Department of Physics, Universitas Airlangga, Surabaya, Indonesia^2

Institute of Biomedical and Health Engineering, Qingshuihe, Shenzhen, China^3

关键词: Average difference;    Communication tools;    Daily life activities;    Directional movements;    Effective communication;    Graphical user interfaces (GUI);    Motion sensors;    Virtual Keyboards;   

Others  :  https://iopscience.iop.org/article/10.1088/1757-899X/602/1/012030/pdf
DOI  :  10.1088/1757-899X/602/1/012030

学科分类：工业工程学

来源: IOP

【 摘 要 】

Individuals with disabilities are often restricted from performing certain daily life activities that the non-disabled persons would normally perform. One of such restrictions is regarding daily life communication that would require listening, speaking, and sometimes movement of one's body part often referred to as a gesture. Thus, disabled persons without fingers and cannot speak fluently finds it challenging to communicate effectively during daily life activities. Therefore, a virtual Graphical User Interface (GUI)-based keyboard communication tool is developed in this study to aid such individuals in their daily life activities. The developed virtual GUI-based keyboard can support the different directional movement of the hand including up, down, left, and right, that were coded as different communication signs. The choice "select" can be done by turning the wrist up and "erasing" by turning the wrist down. Every hand movement is read and translated by a leap motion sensor. There are two kinds of GUI used in this research, the first GUI with the Alphabet key is equipped with a pictorial word while the second GUI Qwerty comes with a pictorial word. Experimental results concerning the effectiveness of the typing effectiveness show the alphabet GUI has a faster typing time than the Qwerty GUI with an average difference of fewer than 15 seconds.