A Spectral-Aware Convolutional Neural Network for Pansharpening

Applied Sciences

卷:10

Jun Li¹  Lin He²  Dahan Xi²  Jiawei Zhu² 

[1] Guangdong Provincial Key Laboratory of Urbanization and Geo-Simulation, Center of Integrated Geographic Information Analysis, School of Geography and Planning, Sun Yat-sen University, Guangzhou 510275, China;

[2] School of Automation Science and Engineering, South China University of Technology, Guangzhou 510640, China;

关键词: pansharpening;    multispectral/hyperspectral image;    spectral-aware;    convolutional neural network;    spatial feature transform;    3D convolution;   

DOI  :  10.3390/app10175809

来源: DOAJ

【 摘 要 】

Pansharpening aims at fusing a low-resolution multiband optical (MBO) image, such as a multispectral or a hyperspectral image, with the associated high-resolution panchromatic (PAN) image to yield a high spatial resolution MBO image. Though having achieved superior performances to traditional methods, existing convolutional neural network (CNN)-based pansharpening approaches are still faced with two challenges: alleviating the phenomenon of spectral distortion and improving the interpretation abilities of pansharpening CNNs. In this work, we develop a novel spectral-aware pansharpening neural network (SA-PNN). On the one hand, SA-PNN employs a network structure composed of a detail branch and an approximation branch, which is consistent with the detail injection framework; on the other hand, SA-PNN strengthens processing along the spectral dimension by using a spectral-aware strategy, which involves spatial feature transforms (SFTs) coupling the approximation branch with the detail branch as well as 3D convolution operations in the approximation branch. Our method is evaluated with experiments on real-world multispectral and hyperspectral datasets, verifying its excellent pansharpening performance.

【 授权许可】

Unknown