CNN and LSTM Based Facial Expression Analysis Model for a Humanoid Robot

IEEE Access

Ping-Huan Kuo¹  Ting-Nan Tsai²  Tzuu-Hseng S. Li²  Po-Chien Luan² 

[1] Computer and Intelligent Robot Program for Bachelor Degree, National Pingtung University, Pingtung, Taiwan;Department of Electrical Engineering, aiRobots Laboratory, National Cheng Kung University, Tainan, Taiwan;

关键词: Convolutional neural network;    long short-term memory;    transfer learning;    facial expression analysis;   

DOI  :  10.1109/ACCESS.2019.2928364

来源: DOAJ

【 摘 要 】

Robots must be able to recognize human emotions to improve the human-robot interaction (HRI). This study proposes an emotion recognition system for a humanoid robot. The robot is equipped with a camera to capture users' facial images, and it uses this system to recognize users' emotions and responds appropriately. The emotion recognition system, based on a deep neural network, learns six basic emotions: happiness, anger, disgust, fear, sadness, and surprise. First, a convolutional neural network (CNN) is used to extract visual features by learning on a large number of static images. Second, a long short-term memory (LSTM) recurrent neural network is used to determine the relationship between the transformation of facial expressions in image sequences and the six basic emotions. Third, CNN and LSTM are combined to exploit their advantages in the proposed model. Finally, the performance of the emotion recognition system is improved by using transfer learning, that is, by transferring knowledge of related but different problems. The performance of the proposed system is verified through leave-one-out cross-validation and compared with that of other models. The system is applied to a humanoid robot to demonstrate its practicability for improving the HRI.

【 授权许可】

Unknown