Study on Representation Invariances of CNNs and Human Visual Information Processing Based on Data Augmentation

Brain Sciences

Linyuan Wang¹  Li Tong¹  Kai Qiao¹  Bin Yan¹  Yibo Cui¹  Chi Zhang¹ 

[1] Henan Key Laboratory of Imaging and Intelligent Processing, PLA Strategic Support Force Information Engineering University, Zhengzhou 450001, China;

关键词: representation invariance;    CNNs;    human visual information processing;    data augmentation;    fMRI visual encoding model;   

DOI  :  10.3390/brainsci10090602

来源: DOAJ

【 摘 要 】

Representation invariance plays a significant role in the performance of deep convolutional neural networks (CNNs) and human visual information processing in various complicated image-based tasks. However, there has been abounding confusion concerning the representation invariance mechanisms of the two sophisticated systems. To investigate their relationship under common conditions, we proposed a representation invariance analysis approach based on data augmentation technology. Firstly, the original image library was expanded by data augmentation. The representation invariances of CNNs and the ventral visual stream were then studied by comparing the similarities of the corresponding layer features of CNNs and the prediction performance of visual encoding models based on functional magnetic resonance imaging (fMRI) before and after data augmentation. Our experimental results suggest that the architecture of CNNs, combinations of convolutional and fully-connected layers, developed representation invariance of CNNs. Remarkably, we found representation invariance belongs to all successive stages of the ventral visual stream. Hence, the internal correlation between CNNs and the human visual system in representation invariance was revealed. Our study promotes the advancement of invariant representation of computer vision and deeper comprehension of the representation invariance mechanism of human visual information processing.

【 授权许可】

Unknown