Intrinsic plasticity via natural gradient descent with application to drift compensation

NEUROCOMPUTING

卷:112

Article

Neumann, K.¹  Strub, C.¹  Steil, J. J.¹ 

[1] Univ Bielefeld, Res Inst Cognit & Robot, CoR Lab, Bielefeld, Germany

关键词: Neural network;    Learning;    Intrinsic plasticity;    Information geometry;    Natural gradient;    Metric tensor;    Gradient descent;    Drift;   

DOI  :  10.1016/j.neucom.2012.12.047

来源: Elsevier

【 摘 要 】

This paper investigates the learning dynamics of intrinsic plasticity (IP), which is a learning rule to tune a neuron's activation function such that its output distribution becomes approximately exponentially distributed. The information-geometric properties of intrinsic plasticity are analyzed and the improved natural gradient intrinsic plasticity (NIP) dynamics are evaluated for a variety of input distributions. Together with a further new modification of the IP rule, the high capability of NIP to cope with drift is demonstrated to have superior performance as compared to the standard gradient in experiments with synthetic and real world data. (C) 2013 Elsevier B.V. All rights reserved.

【 授权许可】

Free