Attention mechanism combined with residual recurrent neural network for sound event detection and localization

EURASIP Journal on Audio, Speech, and Music Processing

Empirical Research

Lei Zhang¹  Yulan Han²  Yuanyuan Zhang²  Chaofeng Lan²  Chao Sun²  Lirong Fu³  Meng Zhang⁴ 

[1] Beidahuang Industry Group General Hospital, 150088, Harbin, People’s Republic of China;Department of School of Measurement and Communication Engineering, Harbin University of Science and Technology, 150080, Harbin, People’s Republic of China;Mechanical and Electrical Engineering College, Hainan University, 570228, Haikou, People’s Republic of China;School of Electronics and Communication Engineering, Guangzhou University, 510006, Guangzhou, People’s Republic of China;

关键词: Sound event;    Detection and localization;    Convolutional cyclic neural network;    Multi-scale feature fusion;    Space channel squeeze excitation module;   

DOI  :  10.1186/s13636-022-00263-6

 received in 2022-03-19, accepted in 2022-11-16,  发布年份 2022

来源: Springer

【 摘 要 】

In the task of sound event detection and localization (SEDL) in a complex environment, the acoustic signals of different events usually have nonlinear superposition, so the detection and localization effect is not good. Given this, this paper is based on the Residual-spatially and channel Squeeze-Excitation (Res-scSE) model. Combined with Multiple-scale Convolutional Recurrent Neural Network (M-CRNN), the Res-scSE-CRNN model is proposed. Firstly, to solve the problem of insufficient extraction of time-frequency feature in single-size convolution kernel, multi-scale feature fusion is carried out by using the feature hierarchy of the convolutional neural network to improve the accuracy of detection. Secondly, aiming at the problem of overlapping audio event localization accuracy is not high, with Res-scSE to replace common convolution module and add residual structure to strengthen the feature extraction, and combining with an attention mechanism to enhance neural network channels and spatial relationships, to improve the network to extract the characteristics of directivity, achieve the goal of the overlapped audio localization. In this paper, experiments are carried out in the open dataset DCASE2019, and evaluation indicators are used to analyze the effectiveness of the proposed model and baseline model in the detection and localization of audio events. The results show that compared with the M-CRNN model, the detection error rate of Res-scSE-CRNN model is reduced 4%, the F1-Score is increased 3.4%, the localization error is reduced by 22.8°, and the frame recall rate is increased 3%.

【 授权许可】

CC BY   
© The Author(s) 2022

【 图 表 】

Fig. 7

Fig. 6

Fig. 1

Fig. 1

Fig. 3

Fig. 1

Fig. 2

Fig. 6

12936_2022_4386_Article_IEq142.gif

Fig. 1

Fig. 1

Fig. 1

Fig. 2

Fig. 2

Fig. 5

Fig. 1

Fig. 4

12936_2022_4386_Article_IEq132.gif

12902_2022_1222_Article_IEq2.gif

Fig. 7

Fig. 4

Fig. 3

Fig. 2

Fig. 6

Fig. 6

Fig. 2

Fig. 4

12936_2022_4386_Article_IEq117.gif

Fig. 3

Fig. 1

Fig. 5

Fig. 6

13690_2022_1011_Article_IEq4.gif

13690_2022_1011_Article_IEq3.gif

13690_2022_1011_Article_IEq2.gif

13690_2022_1011_Article_IEq1.gif

Fig. 7

Fig. 2

Fig. 1

Fig. 1

Fig. 1

40644_2022_507_Article_IEq1.gif

Fig. 3

Fig. 4

12936_2022_4386_Article_IEq84.gif

Fig. 2

Fig. 2

Fig. 1

Fig. 6

Fig. 1

12902_2022_1244_Article_IEq30.gif

Fig. 5

12902_2022_1244_Article_IEq28.gif

12902_2022_1244_Article_IEq27.gif

Fig. 4

12902_2022_1244_Article_IEq24.gif

12902_2022_1244_Article_IEq23.gif

12902_2022_1244_Article_IEq22.gif

12902_2022_1244_Article_IEq21.gif

12902_2022_1244_Article_IEq19.gif

Fig.3

12902_2022_1244_Article_IEq17.gif

Fig. 3

Fig. 2

Fig. 1

Fig. 2

Fig. 2

Fig. 1

Fig. 3

12902_2022_1244_Article_IEq8.gif

Fig. 4

12982_2022_119_Article_IEq182.gif

12982_2022_119_Article_IEq181.gif

12982_2022_119_Article_IEq176.gif

12982_2022_119_Article_IEq173.gif

12982_2022_119_Article_IEq171.gif

12982_2022_119_Article_IEq169.gif

12888_2022_4451_Article_IEq2.gif

12982_2022_119_Article_IEq164.gif

12982_2022_119_Article_IEq162.gif

12982_2022_119_Article_IEq160.gif

12982_2022_119_Article_IEq158.gif

12982_2022_119_Article_IEq156.gif