Multimodal Emotion Recognition Using the Symmetric S-ELM-LUPI Paradigm

Symmetry

Michele Mukeshimana¹  Zhe Chen²  Xiaojuan Ban³  Lingzhi Yang³ 

[1] Faculity of Engineering Sciences, University of Burundi, P.O. Box 1550 Bujumbura, Burundi;Qingdao Hisense Group Co., Ltd., Qingdao 266000, China;School of Computer and Communication Engineering, Beijing Key Laboratory of Knowledge Engineering for Materials Science, University of Science and Technology Beijing, Beijing 10083, China;

关键词: multimodal emotion recognition;    symmetric S-ELM-LUPI paradigm;    human-machine interaction;   

DOI  :  10.3390/sym11040487

来源: DOAJ

【 摘 要 】

Multimodal emotion recognition has become one of the new research fields of human-machine interaction. This paper focuses on feature extraction and data fusion in audio-visual emotion recognition, aiming at improving recognition effect and saving storage space. A semi-serial fusion symmetric method is proposed to fuse the audio and visual patterns of emotional recognition, and a method of Symmetric S-ELM-LUPI is adopted (Symmetric Sparse Extreme Learning Machine-Learning Using Privileged Information). The method inherits the generalized high speed of the Extreme Learning Machine, and combines this with the acceleration in the recognition process by the Learning Using Privileged Information and the memory saving of the Sparse Extreme Learning Machine. It is a learning method, which improves the traditional learning methods of examples and targets only. It introduces the role of a teacher in providing additional information to enhance the recognition (test) without complicating the learning process. The proposed method is tested on publicly available datasets and yields promising results. This method regards one pattern as the standard information source, while the other pattern as the privileged information source. Each mode can be treated as privileged information for another mode. The results show that this method is appropriate for multi-modal emotion recognition. For hundreds of samples, the execution time is less than one percent seconds. The sparsity of the proposed method has the advantage of storing memory economy. Compared with other machine learning methods, this method is more accurate and stable.

【 授权许可】

Unknown