A Novel Algorithm for Acoustic and Visual Classifiers Decision Fusion in Audio-Visual Speech Recognition System

Signal Processing: An International Journal

P.S. Sathidevi¹  Rajavel¹ 

[1] $$

关键词: Audio-visual speech recognition;    Reliability-ratio based weight optimization;    late integration;   

DOI  :  

学科分类：物理（综合）

来源: Computer Science Journals

【 摘 要 】

Audio-visual speech recognition (AVSR) using acoustic and visual signals of speech have received attention recently because of its robustness in noisy environments. Perceptual studies also support this approach by emphasizing the importance of visual information for speech recognition in humans. An importantissue in decision fusion based AVSR system is how to obtain the appropriate integration weight for the speech modalities to integrate and ensurethe combined AVSR system’s performances better than that of the audio-only and visual-only systems under various noise conditions. To solve this issue, we present a genetic algorithm (GA) based optimization scheme to obtain the appropriate integration weight from the relative reliability of each modality. The performance of the proposed GA optimized reliability-ratio based weight estimation scheme is demonstrated via single speaker, mobile functions isolatedword recognition experiments. The results show that the proposed scheme improves robust recognition accuracy over the conventional unimodal systems and the baseline reliability ratio-based AVSR system under various signal to noise ratio conditions.

【 授权许可】

Unknown