Information Decomposition and Synergy

Entropy

Eckehard Olbrich¹  Nils Bertschinger³  Johannes Rauh² 

[1] Max Planck Institute for Mathematics in the Sciences, Inselstraße 22, 04103 Leipzig, Germany;Institute of Algebraic Geometry, Leibniz Universität Hannover, Welfengarten 1, 30167 Hannover, Germany; E-Mail:;Frankfurt Institute for Advanced Studies, Ruth-Moufang-Straße 1, 60438 Frankfurt am Main, Germany; E-Mail:

关键词: Shannon information;    mutual information;    information decomposition;    shared information;    synergy;   

DOI  :  10.3390/e17053501

来源: mdpi

【 摘 要 】

Recently, a series of papers addressed the problem of decomposing the information of two random variables into shared information, unique information and synergistic information. Several measures were proposed, although still no consensus has been reached. Here, we compare these proposals with an older approach to define synergistic information based on the projections on exponential families containing only up to k-th order interactions. We show that these measures are not compatible with a decomposition into unique, shared and synergistic information if one requires that all terms are always non-negative (local positivity). We illustrate the difference between the two measures for multivariate Gaussians.

【 授权许可】

CC BY   
© 2015 by the authors; licensee MDPI, Basel, Switzerland