See clearly on rainy days: Hybrid multiscale loss guided multi-feature fusion network for single image rain removal

Computational Visual Media

Yu Zhang¹  Huadong Ma¹  Huiyuan Fu¹ 

[1] Beijing Key Laboratory of Intelligent Telecommunications Software and Multimedia, Beijing University of Posts and Telecommunications, 100876, Beijing, China;

关键词: single image rain removal;    multiple feature fusion;    deep learning;    hybrid multiscale loss;   

DOI  :  10.1007/s41095-021-0210-3

来源: Springer

【 摘 要 】

The quality of photos is highly susceptible to severe weather such as heavy rain; it can also degrade the performance of various visual tasks like object detection. Rain removal is a challenging problem because rain streaks have different appearances even in one image. Regions where rain accumulates appear foggy or misty, while rain streaks can be clearly seen in areas where rain is less heavy. We propose removing various rain effects in pictures using a hybrid multiscale loss guided multiple feature fusion de-raining network (MSGMFFNet). Specially, to deal with rain streaks, our method generates a rain streak attention map, while preprocessing uses gamma correction and contrast enhancement to enhanced images to address the problem of rain accumulation. Using these tools, the model can restore a result with abundant details. Furthermore, a hybrid multiscale loss combining L1 loss and edge loss is used to guide the training process to pay attention to edge and content information. Comprehensive experiments conducted on both synthetic and real-world datasets demonstrate the effectiveness of our method.

【 授权许可】

CC BY