A fully trainable network with RNN-based pooling

NEUROCOMPUTING

卷:338

Article

Li, Shuai^1,2  Li, Wanqing¹  Cook, Chris¹  Zhu, Ce²  Gao, Yanbo² 

[1] Univ Wollongong, Sch Comp & Informat Technol, Wollongong, NSW 2522, Australia

[2] Univ Elect Sci & Technol China, Sch Elect Engn, Chengdu 611731, Sichuan, Peoples R China

关键词: Pooling;    Recurrent neural network;    Convolutional neural network;    Deep learning;   

DOI  :  10.1016/j.neucom.2019.02.004

来源: Elsevier

【 摘 要 】

Pooling is an important component in convolutional neural networks (CNNs) for aggregating features and reducing computational burden. Compared with other components such as convolutional layers and fully connected layers which are completely learned from data, the pooling component is still handcrafted such as max pooling and average pooling. This paper proposes a learnable pooling function using recurrent neural networks (RNN) so that the pooling can be fully adapted to data and other components of the network, leading to an improved performance. Such a network with learnable pooling function is referred to as a fully trainable network (FTN). Experimental results demonstrate that the proposed RNN based pooling can well approximate the existing pooling functions with just one neuron, thus making it appropriate to be used as pooling function in a network with its rich representation capability. Furthermore, experiments have shown that the proposed FTN can achieve better performance than the existing pooling methods under similar network architectures for image classification. (C) 2019 Elsevier B.V. All rights reserved.

【 授权许可】

Free