A neural network-supported two-stage algorithm for lightweight dereverberation on hearing devices

EURASIP Journal on Audio, Speech, and Music Processing

Empirical Research

Raphael Koning¹  Joachim Thiemann¹  Jean-Marie Lemercier²  Timo Gerkmann² 

[1] Advanced Bionics, Hannover, Germany;Signal Processing, Universität Hamburg, Hamburg, Germany;

关键词: Dereverberation;    Neural network;    End-to-end learning;    Hearing devices;   

DOI  :  10.1186/s13636-023-00285-8

 received in 2022-11-24, accepted in 2023-04-11,  发布年份 2023

来源: Springer

【 摘 要 】

A two-stage lightweight online dereverberation algorithm for hearing devices is presented in this paper. The approach combines a multi-channel multi-frame linear filter with a single-channel single-frame post-filter. Both components rely on power spectral density (PSD) estimates provided by deep neural networks (DNNs). By deriving new metrics analyzing the dereverberation performance in various time ranges, we confirm that directly optimizing for a criterion at the output of the multi-channel linear filtering stage results in a more efficient dereverberation as compared to placing the criterion at the output of the DNN to optimize the PSD estimation. More concretely, we show that training this stage end-to-end helps further remove the reverberation in the range accessible to the filter, thus increasing the early-to-moderate reverberation ratio. We argue and demonstrate that it can then be well combined with a post-filtering stage to efficiently suppress the residual late reverberation, thereby increasing the early-to-final reverberation ratio. This proposed two-stage procedure is shown to be both very effective in terms of dereverberation performance and computational demands, as compared to, e.g., recent state-of-the-art DNN approaches. Furthermore, the proposed two-stage system can be adapted to the needs of different types of hearing-device users by controlling the amount of reduction of early reflections.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

41116_2023_36_Article_IEq672.gif

41116_2023_36_Article_IEq670.gif

41116_2023_36_Article_IEq664.gif

41116_2023_36_Article_IEq663.gif

41116_2023_36_Article_IEq658.gif

41116_2023_36_Article_IEq657.gif

41116_2023_36_Article_IEq656.gif

41116_2023_36_Article_IEq655.gif

41116_2023_36_Article_IEq653.gif

41116_2023_36_Article_IEq649.gif

41116_2023_36_Article_IEq644.gif

Fig. 2

Fig. 1

41116_2023_36_Article_IEq633.gif

41116_2023_36_Article_IEq631.gif

41116_2023_36_Article_IEq627.gif

41116_2023_36_Article_IEq623.gif

41116_2023_36_Article_IEq619.gif

41116_2023_36_Article_IEq616.gif

Fig. 1

40517_2023_258_Article_IEq11.gif

Fig. 1

Fig. 4

41116_2023_36_Article_IEq508.gif

40798_2023_579_Article_IEq5.gif

41116_2023_36_Article_IEq506.gif

41116_2023_36_Article_IEq505.gif

40798_2023_579_Article_IEq3.gif

41116_2023_36_Article_IEq504.gif

41116_2023_36_Article_IEq503.gif

41116_2023_36_Article_IEq502.gif

41116_2023_36_Article_IEq501.gif

Fig. 7

Fig. 2

Fig. 2

40517_2023_258_Article_IEq8.gif

40517_2023_258_Article_IEq7.gif

Fig. 1

41116_2023_36_Article_IEq500.gif

41116_2023_36_Article_IEq499.gif

41116_2023_36_Article_IEq498.gif

41116_2023_36_Article_IEq497.gif

41116_2023_36_Article_IEq496.gif

Fig. 1

41116_2023_36_Article_IEq493.gif

12888_2023_4776_Article_IEq1.gif

41116_2023_36_Article_IEq490.gif

41116_2023_36_Article_IEq489.gif

41116_2023_36_Article_IEq488.gif

41116_2023_36_Article_IEq487.gif

41116_2023_36_Article_IEq486.gif

Fig. 4

41116_2023_36_Article_IEq484.gif

41116_2023_36_Article_IEq483.gif

41116_2023_36_Article_IEq482.gif

Fig. 3

41116_2023_36_Article_IEq479.gif

41116_2023_36_Article_IEq478.gif

Fig. 3

41116_2023_36_Article_IEq476.gif

41116_2023_36_Article_IEq475.gif

Fig. 3

Fig. 3

41116_2023_36_Article_IEq473.gif

Fig. 2

41116_2023_36_Article_IEq470.gif

41116_2023_36_Article_IEq469.gif

41116_2023_36_Article_IEq468.gif

41116_2023_36_Article_IEq467.gif

41116_2023_36_Article_IEq466.gif

Fig. 4

41116_2023_36_Article_IEq465.gif

41116_2023_36_Article_IEq464.gif

Fig. 2

Fig. 1

41116_2023_36_Article_IEq461.gif

Fig. 1

41116_2023_36_Article_IEq459.gif

41116_2023_36_Article_IEq458.gif

41116_2023_36_Article_IEq457.gif

41116_2023_36_Article_IEq456.gif

41116_2023_36_Article_IEq455.gif

41116_2023_36_Article_IEq454.gif

41116_2023_36_Article_IEq453.gif

41116_2023_36_Article_IEq452.gif

41116_2023_36_Article_IEq450.gif

Fig. 2

41116_2023_36_Article_IEq449.gif

41116_2023_36_Article_IEq448.gif

41116_2023_36_Article_IEq447.gif

41116_2023_36_Article_IEq446.gif

40517_2023_255_Article_IEq1.gif

41116_2023_36_Article_IEq445.gif

41116_2023_36_Article_IEq444.gif

Fig. 3

Fig. 3

41116_2023_36_Article_IEq437.gif

41116_2023_36_Article_IEq436.gif

41116_2023_36_Article_IEq435.gif

41116_2023_36_Article_IEq434.gif

Fig. 3

Fig. 2

41116_2023_36_Article_IEq431.gif

41116_2023_36_Article_IEq430.gif

41116_2023_36_Article_IEq429.gif

Fig. 2

41116_2023_36_Article_IEq427.gif

41116_2023_36_Article_IEq426.gif

41116_2023_36_Article_IEq425.gif

Fig. 1

41116_2023_36_Article_IEq424.gif

41116_2023_36_Article_IEq423.gif

41116_2023_36_Article_IEq421.gif

41116_2023_36_Article_IEq420.gif

41116_2023_36_Article_IEq419.gif

41116_2023_36_Article_IEq418.gif

41116_2023_36_Article_IEq417.gif

41116_2023_36_Article_IEq416.gif

Fig. 1

41116_2023_36_Article_IEq414.gif

41116_2023_36_Article_IEq413.gif

41116_2023_36_Article_IEq412.gif

41116_2023_36_Article_IEq410.gif

41116_2023_36_Article_IEq408.gif

41116_2023_36_Article_IEq407.gif

Fig. 3

41116_2023_36_Article_IEq401.gif

Fig. 2

41116_2023_36_Article_IEq399.gif

41116_2023_36_Article_IEq398.gif

41116_2023_36_Article_IEq397.gif

41116_2023_36_Article_IEq396.gif

Fig. 1

41116_2023_36_Article_IEq393.gif

41116_2023_36_Article_IEq392.gif

41116_2023_36_Article_IEq391.gif

41116_2023_36_Article_IEq390.gif

41116_2023_36_Article_IEq389.gif

41116_2023_36_Article_IEq388.gif

41116_2023_36_Article_IEq387.gif

41116_2023_36_Article_IEq386.gif

41116_2023_36_Article_IEq385.gif

41116_2023_36_Article_IEq384.gif

Fig. 1

41116_2023_36_Article_IEq358.gif

41116_2023_36_Article_IEq338.gif

41116_2023_36_Article_IEq310.gif

41116_2023_36_Article_IEq291.gif

41116_2023_36_Article_IEq282.gif