Nonlinear reconfiguration of network edges, topology and information content during an artificial learning task

Brain Informatics

Oluwasanmi Koyejo¹  James M. Shine²  Mike Li³  Joseph T. Lizier⁴  Ben Fulcher⁵ 

[1] Beckman Institute for Advanced Science and Technology, University of Illinois Champaign, Champaign, USA;Department of Computer Science, University of Illinois at Urbana-Champaign, Champaign, USA;Centre for Complex Systems, The University of Sydney, 2006, Camperdown, NSW, Australia;Brain and Mind Centre, The University of Sydney, 2050, Camperdown, NSW, Australia;Centre for Complex Systems, The University of Sydney, 2006, Camperdown, NSW, Australia;Brain and Mind Centre, The University of Sydney, 2050, Camperdown, NSW, Australia;Complex Systems Research Group, Faculty of Engineering, The University of Sydney, 2006, Camperdown, NSW, Australia;Centre for Complex Systems, The University of Sydney, 2006, Camperdown, NSW, Australia;Complex Systems Research Group, Faculty of Engineering, The University of Sydney, 2006, Camperdown, NSW, Australia;Centre for Complex Systems, The University of Sydney, 2006, Camperdown, NSW, Australia;School of Physics, The University of Sydney, 2006, Camperdown, NSW, Australia;

关键词: Network;    Integration;    Low-dimensional;    Information;    Pixels;    Reconfiguration;    Systems;   

DOI  :  10.1186/s40708-021-00147-z

来源: Springer

【 摘 要 】

Here, we combine network neuroscience and machine learning to reveal connections between the brain’s network structure and the emerging network structure of an artificial neural network. Specifically, we train a shallow, feedforward neural network to classify hand-written digits and then used a combination of systems neuroscience and information-theoretic tools to perform ‘virtual brain analytics’ on the resultant edge weights and activity patterns of each node. We identify three distinct phases of network reconfiguration across learning, each of which are characterized by unique topological and information-theoretic signatures. Each phase involves aligning the connections of the neural network with patterns of information contained in the input dataset or preceding layers (as relevant). We also observe a process of low-dimensional category separation in the network as a function of learning. Our results offer a systems-level perspective of how artificial neural networks function—in terms of multi-stage reorganization of edge weights and activity patterns to effectively exploit the information content of input data during edge-weight training—while simultaneously enriching our understanding of the methods used by systems neuroscience.

【 授权许可】

CC BY