Mutual information against correlations in binary communication channels

BMC Neuroscience

Eligiusz Wajnryb¹  Janusz Szczepanski¹  Agnieszka Pregowska¹ 

[1] Institute of Fundamental Technological Research, Polish Academy of Sciences, Pawinskiego 5BWarsaw, PL

关键词: Neuronal encoding;    Correlation;    Mutual information;    Entropy;    Communication channel;    Shannon information;   

Others  :  1210186
DOI  :  10.1186/s12868-015-0168-0

 received in 2014-11-22, accepted in 2015-04-21,  发布年份 2015

【 摘 要 】

Background

Explaining how the brain processing is so fast remains an open problem (van Hemmen JL, Sejnowski T., 2004). Thus, the analysis of neural transmission (Shannon CE, Weaver W., 1963) processes basically focuses on searching for effective encoding and decoding schemes. According to the Shannon fundamental theorem, mutual information plays a crucial role in characterizing the efficiency of communication channels. It is well known that this efficiency is determined by the channel capacity that is already the maximal mutual information between input and output signals. On the other hand, intuitively speaking, when input and output signals are more correlated, the transmission should be more efficient. A natural question arises about the relation between mutual information and correlation. We analyze the relation between these quantities using the binary representation of signals, which is the most common approach taken in studying neuronal processes of the brain.

Results

We present binary communication channels for which mutual information and correlation coefficients behave differently both quantitatively and qualitatively. Despite this difference in behavior, we show that the noncorrelation of binary signals implies their independence, in contrast to the case for general types of signals.

Conclusions

Our research shows that the mutual information cannot be replaced by sheer correlations. Our results indicate that neuronal encoding has more complicated nature which cannot be captured by straightforward correlations between input and output signals once the mutual information takes into account the structure and patterns of the signals.

【 授权许可】

   
2015 Pregowska et al.; licensee BioMed Central.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

【 参考文献 】

• [1]van Hemmen JL, Sejnowski T: 23 Problems in Systems Neurosciences. Oxford University Press, UK; 2006.
• [2]Shannon CE, Weaver W: The Mathematical Theory of Communication. University of Illinois Press, Urbana, United States of America; 1963.
• [3]Shannon CE: A mathematical theory of communication. Bell Syst Tech J. 1948, 27:379-423623656.
• [4]Borst JL, Theunissen FE: Information theory and neural coding. Nat Neurosci. 1999, 2:947-57.
• [5]Paprocki B, Szczepanski J: Transmission efficiency in ring, brain inspired neuronal networks. information and energetic aspects. Brain Res. 2013, 1536:135-43.
• [6]Cohen MR, Kohn A: Measuring and interpreting neuronal correlations. Nat Neurosci. 2011, 14:811-9.
• [7]Arnold M, Szczepanski J, Montejo N, Wajnryb E, Sanchez-Vives MV: Information content in cortical spike trains during brain state transitions. J Sleep Res. 2013, 2:13-21.
• [8]Abbott LF, Dayan P: The effect of correlated variability on the accuracy of a population code. Neural Comput. 1999, 11:91-101.
• [9]Nirenberg S, Latham PE. Decoding neuronal spike trains: how important are correlations? In: Proceedings of National Academy of Science USA, 10 June 2003. National Academy of Science USA: 2003. p. 7348–53.
• [10]de la Rocha J, Doiron B, Shea-Brown E, Josic K, Reyes A: Correlation between neural spike trains increases with firing rate. Nature 2007, 448:802-6.
• [11]Pillow JW, Shlens J, Paninski L, Sher A, Litke AM, Chicgilnisky J, et al.: Spatio-temporal correlations and visual signaling in a complete neuronal population. Nature 2008, 454:995-9.
• [12]Amari S: Measure of correlation orthogonal to change in firing rate. Neural Comput. 2009, 21:960-72.
• [13]Ecker AS, Berens P, Keliris GA, Bethge M, Logothetis NK, Tolias AS: Decorrelated neuronal firing in cortical microcircuits. Science 2010, 327:584-7.
• [14]Nienborg H, Cumming B: Stimulus correlations between the activity of sensory neurons and behavior: how much do they tell us about a neuron’s causality? Curr Opin Neurobiology 2010, 20:376-81.
• [15]Feller W: An Introduction to Probability Theory and Its Applications. A Wiley Publications in Statistics, New York, United States of America; 1958.
• [16]Kohn A, Smith MA: Stimulus dependence of neuronal correlation in primary visual cortex of the macaque. J Neurosci. 2005, 25:3661-73.
• [17]Ash RB: The Mathematical Theory of Communication. John Wiley and Sons, New York, London, Sydney, United States of America; 1965.
• [18]Eguia MC, Rabinovich MI, Abarbanel HDI: Information transmissionand recovery in neural communications channels. Phys Rev E 2000, 65(5):7111-22.
• [19]Moreno-Bote R, Beck J, Kanitscheider I, Pitkow X, Latham P, Pouget A: Information-limiting correlations. Nat Neurosci. 2014, 17:1410-7.
• [20]Pitkow X, Meister M: Decorrelation and efficient coding by retinal ganglion cells. Nat Neurosci. 2012, 15:628-35.
• [21]Cover TM, Thomas JA: Elements of Information Theory. A Wiley-Interscience Publication, New York, United States of America; 1991.
• [22]Rolls ET, Aggelopoulos NC, Franco L, Treves A: Information encoding in the inferior temporal visual cortex: contributions of the firing rates and the correlations between the firing of neurons. Biol Cybernetics 2004, 90:19-32.
• [23]Levin JE, Miller JP: Broadband neural encoding in the cricket cercal sensory system enhanced by stochastic resonance. Nature 2004, 380:165-8.
• [24]Amigo JM, Szczepanski J, Wajnryb E, Sanchez-Vives MV: Estimating the entropy rate of spike trains via lempel-ziv complexity. Neural Comput. 2004, 16:717-36.
• [25]Chapeau–Blondeau F, Rousseau D, Delahaines A: Renyi entropy measure of noise-aided information transmission in a binary channel. Phys Rev E 2010, 81(051112):1-10.
• [26]DeWesse MR, Wehr M, Zador A: Binary spiking in auditory cortex. J Neurosci. 2003, 27(23/21):7940-9.
• [27]Paninski L: Estimation of entropy and mutual information. Neural Comput. 2003, 15(6):1191-253.
• [28]London M, Larkum ME, Hausser M: Predicting the synaptic information efficacy in cortical layer 5 pyramidal neurons using a minimal integrate-and-fire model. Biol Cybernetics 2008, 99:393-401.
• [29]Kraskov A, Stogbauer H, Grassberger P: Estimating mutual information. Phys Rev E 2004, 69(6):066138.
• [30]Panzeri S, Schultz SR, Treves A, Rolls ET: Correlation and the encoding of information in the nervous system. Proc R Soc London 1999, B:1001-12.
• [31]Lempel A, Ziv J: On the complexity of finite sequences. IEEE Trans Inf Theory 1976, 22(1):75-81.
• [32]Lempel A, Ziv J: On the complexity of individual sequences. IEEE Trans Inf Theory 1976, IT-22:75.
• [33]Strong SP, Koberle R, de Ruyter van Steveninck RR, Bialek W: Entropy and information in neural spike trains. Phys Rev Lett. 1998, 80(1):197-200.
• [34]Szczepanski J, Arnold M, Wajnryb E, Amigo JM, Sanchez-Vives MV: Mutual information and redundancy in spontaneous communication between cortical neurons. Biol Cybernetics 2011, 104:161-74.
• [35]Bair W, Zohary E, Newsome WT: Correlated firing in macaque visual area mt: time scales and relationship to behavior. J Neurosci. 2001, 21(5):1676-97.