Channel and temporal-frequency attention UNet for monaural speech enhancement

EURASIP Journal on Audio, Speech, and Music Processing

Empirical Research

Shiyun Xu¹  Zehua Zhang¹  Mingjiang Wang¹ 

[1] Key Laboratory for Key Technologies of IoT Terminals, Harbin Institute of Technology, Shenzhen, China;

关键词: Speech enhancement;    Neural network;    Denoising;    Dereverberation;   

DOI  :  10.1186/s13636-023-00295-6

 received in 2023-03-17, accepted in 2023-07-24,  发布年份 2023

来源: Springer

【 摘 要 】

The presence of noise and reverberation significantly impedes speech clarity and intelligibility. To mitigate these effects, numerous deep learning-based network models have been proposed for speech enhancement tasks aimed at improving speech quality. In this study, we propose a monaural speech enhancement model called the channel and temporal-frequency attention UNet (CTFUNet). CTFUNet takes the noisy spectrum as input and produces a complex ideal ratio mask (cIRM) as output. To improve the speech enhancement performance of CTFUNet, we employ multi-scale temporal-frequency processing to extract input speech spectrum features. We also utilize multi-conv head channel attention and residual channel attention to capture temporal-frequency and channel features. Moreover, we introduce the channel temporal-frequency skip connection to alleviate information loss between down-sampling and up-sampling. On the blind test set of the first deep noise suppression challenge, our proposed CTFUNet has better denoising performance than the champion models and the latest models. Furthermore, our model outperforms recent models such as Uformar and MTFAA in both denoising and dereverberation performance.

【 授权许可】

CC BY   
© Springer Nature Switzerland AG 2023

【 图 表 】

13068_2023_2361_Article_IEq1.gif

42490_2023_74_Article_IEq59.gif

13690_2023_1172_Article_IEq8.gif

13690_2023_1172_Article_IEq7.gif

13690_2023_1172_Article_IEq6.gif

13690_2023_1172_Article_IEq5.gif

Fig. 1

Fig. 3

13690_2023_1172_Article_IEq2.gif

13690_2023_1172_Article_IEq1.gif

Fig. 1

Fig. 1

Fig. 1

Fig. 5

Fig. 1

Fig. 6

Fig. 4

Fig. 4

Fig. 1

Fig. 2

Fig. 3

Fig. 1

Fig. 1

Fig. 2

42490_2023_74_Article_IEq42.gif

Fig. 4

42490_2023_74_Article_IEq40.gif

42490_2023_74_Article_IEq39.gif

42490_2023_74_Article_IEq38.gif

Fig. 3

42490_2023_74_Article_IEq37.gif

42490_2023_74_Article_IEq36.gif

42490_2023_74_Article_IEq35.gif

Fig. 3

Fig. 9

Fig. 1

Fig. 1

42490_2023_74_Article_IEq30.gif

42490_2023_74_Article_IEq29.gif

Fig. 3

42490_2023_74_Article_IEq27.gif

42490_2023_74_Article_IEq25.gif

42490_2023_74_Article_IEq24.gif

42490_2023_74_Article_IEq23.gif

Fig. 1

Fig. 8

42490_2023_74_Article_IEq20.gif

42490_2023_74_Article_IEq19.gif

42490_2023_74_Article_IEq18.gif

42490_2023_74_Article_IEq17.gif

42490_2023_74_Article_IEq13.gif

Fig. 2

Fig. 7

42490_2023_74_Article_IEq8.gif

Fig. 1

Fig. 1

Fig. 6

Fig. 1

Fig. 6

42490_2023_74_Article_IEq3.gif

42490_2023_74_Article_IEq1.gif

Fig. 2

Fig. 3

Fig. 2

Fig. 6

Fig. 5

Fig. 10

Fig. 1

Fig. 5

Fig. 7

Fig. 8

Fig. 5

Fig. 4

Fig. 1

Fig. 2

Fig. 3

Fig. 5

Fig. 1

Fig. 6

Fig. 2

Fig. 4

Fig. 1

Fig. 1

Fig. 4

Fig. 1

Fig. 3

Fig. 7

Fig. 3

Fig. 3

Fig. 6

Fig. 6

Fig. 2

Fig. 1

Fig. 5

Fig. 2

Fig. 1

Fig. 4

Fig. 2

40798_2023_622_Article_IEq21.gif

40798_2023_622_Article_IEq19.gif

Fig. 6

Fig. 9

Fig. 1

Fig. 5

40798_2023_622_Article_IEq15.gif

40517_2023_263_Article_IEq1.gif

Fig. 10

Fig. 5

Scheme 1

Fig. 3

40798_2023_622_Article_IEq9.gif

Fig. 2

Fig. 5

Fig. 4

12888_2023_4959_Article_IEq3.gif

Fig. 4

12888_2023_4959_Article_IEq1.gif

Fig. 8

Fig. 6

Fig. 4

Fig. 8

Fig. 3

Fig. 11

Fig. 4

Fig. 12

Fig. 1

Fig. 3

Fig. 3

Fig. 1

Fig. 1

12938_2023_1137_Article_IEq68.gif

Fig. 5

40798_2023_622_Article_IEq7.gif

40798_2023_622_Article_IEq6.gif

40798_2023_622_Article_IEq4.gif

Fig. 5

Fig. 9

40798_2023_622_Article_IEq1.gif

Fig. 2

41512_2023_153_Article_IEq116.gif

Fig. 2

Fig. 1

Fig. 3

Fig. 3

Fig. 3

41512_2023_153_Article_IEq93.gif

Fig. 6