Infrared and Visible Image Fusion with a Generative Adversarial Network and a Residual Network

Applied Sciences

Shuyan Xu¹  Yongcheng Wang¹  Xin Zhang¹  Dongdong Xu¹  Ning Zhang¹  Kaiguang Zhu² 

[1] Changchun Institute of Optics, Fine Mechanics and Physics, Chinese Academy of Sciences, Changchun 130033, China;Key Laboratory of Geo-Exploration Instrumentation, Ministry of Education, Jilin University, Changchun 130033, China;

关键词: infrared and visible image fusion;    deep learning;    generative adversarial network;    residual network;    structural similarity loss;   

DOI  :  10.3390/app10020554

来源: DOAJ

【 摘 要 】

Infrared and visible image fusion can obtain combined images with salient hidden objectives and abundant visible details simultaneously. In this paper, we propose a novel method for infrared and visible image fusion with a deep learning framework based on a generative adversarial network (GAN) and a residual network (ResNet). The fusion is accomplished with an adversarial game and directed by the unique loss functions. The generator with residual blocks and skip connections can extract deep features of source image pairs and generate an elementary fused image with infrared thermal radiation information and visible texture information, and more details in visible images are added to the final images through the discriminator. It is unnecessary to design the activity level measurements and fusion rules manually, which are now implemented automatically. Also, there are no complicated multi-scale transforms in this method, so the computational cost and complexity can be reduced. Experiment results demonstrate that the proposed method eventually gets desirable images, achieving better performance in objective assessment and visual quality compared with nine representative infrared and visible image fusion methods.

【 授权许可】

Unknown