Deepsign: Sign Language Detection and Recognition Using Deep Learning

Electronics

Juan M. Corchado¹  Ana-Belén Gil-González¹  Chintan Bhatt²  Kevin Patel²  Krenil Sapariya²  Deep Kothadiya² 

[1] BISITE Research Group, University of Salamanca, 37007 Salamanca, Spain;U & P U Patel Department of Computer Engineering, CSPIT, CHARUSAT Campus, Charotar University of Science and Technology (CHARUSAT), Changa 388421, India;

关键词: Indian sign language;    deep learning;    LSTM;    GRU;    sign;   

DOI  :  10.3390/electronics11111780

来源: DOAJ

【 摘 要 】

The predominant means of communication is speech; however, there are persons whose speaking or hearing abilities are impaired. Communication presents a significant barrier for persons with such disabilities. The use of deep learning methods can help to reduce communication barriers. This paper proposes a deep learning-based model that detects and recognizes the words from a person’s gestures. Deep learning models, namely, LSTM and GRU (feedback-based learning models), are used to recognize signs from isolated Indian Sign Language (ISL) video frames. The four different sequential combinations of LSTM and GRU (as there are two layers of LSTM and two layers of GRU) were used with our own dataset, IISL2020. The proposed model, consisting of a single layer of LSTM followed by GRU, achieves around 97% accuracy over 11 different signs. This method may help persons who are unaware of sign language to communicate with persons whose speech or hearing is impaired.

【 授权许可】

Unknown