Natural Gradient Flow in the Mixture Geometry of a Discrete Exponential Family†

Entropy

Luigi Malagò²  Giovanni Pistone³  Frຝéric Barbaresco¹ 

[1] id="af1-entropy-17-04215">1Department of Electrical and Electronic Engineering, Shinshu University, Nagano, JapanJapan;De Castro Statistics, Collegio Carlo Alberto, Moncalieri, Italy; E-Mail:

关键词: information geometry;    stochastic relaxation;    natural gradient flow;    expectation parameters;    toric models;   

DOI  :  10.3390/e17064215

来源: mdpi

【 摘 要 】

In this paper, we study Amari’s natural gradient flows of real functions defined on the densities belonging to an exponential family on a finite sample space. Our main example is the minimization of the expected value of a real function defined on the sample space. In such a case, the natural gradient flow converges to densities with reduced support that belong to the border of the exponential family. We have suggested in previous works to use the natural gradient evaluated in the mixture geometry. Here, we show that in some cases, the differential equation can be extended to a bigger domain in such a way that the densities at the border of the exponential family are actually internal points in the extended problem. The extension is based on the algebraic concept of an exponential variety. We study in full detail a toy example and obtain positive partial results in the important case of a binary sample space.

【 授权许可】

CC BY   
© 2015 by the authors; licensee MDPI, Basel, Switzerland