Knowledge Distillation of Grassmann Manifold Network for Remote Sensing Scene Classification

Remote Sensing

Deshi Li¹  Ling Tian¹  Bokun He¹  Chu He¹  Dingwen Wang²  Zhichao Wang² 

[1] Electronic Information School, Wuhan University, Wuhan 430072, China;School of Computer Science, Wuhan University, Wuhan 430072, China;

关键词: knowledge distillation;    Grassmann manifold;    neural network;   

DOI  :  10.3390/rs13224537

来源: DOAJ

【 摘 要 】

Due to device limitations, small networks are necessary for some real-world scenarios, such as satellites and micro-robots. Therefore, the development of a network with both good performance and small size is an important area of research. Deep networks can learn well from large amounts of data, while manifold networks have outstanding feature representation at small sizes. In this paper, we propose an approach that exploits the advantages of deep networks and shallow Grassmannian manifold networks. Inspired by knowledge distillation, we use the information learned from convolutional neural networks to guide the training of the manifold networks. Our approach leads to a reduction in model size, which addresses the problem of deploying deep learning on resource-limited embedded devices. Finally, a series of experiments were conducted on four remote sensing scene classification datasets. The method in this paper improved the classification accuracy by 2.31% and 1.73% on the UC Merced Land Use and SIRIWHU datasets, respectively, and the experimental results demonstrate the effectiveness of our approach.

【 授权许可】

Unknown