Entropy Measures vs. Kolmogorov Complexity

Entropy

Andreia Teixeira¹  Armando Matos¹  André Souto¹ 

[1] Computer Science Department, Faculty of Sciences, University of Porto, Rua Campo Alegre 1021/1055, 4169-007 Porto, Portugal; E-Mails:

关键词: Kolmogorov complexity;    Shannon entropy;    Rényi entropy;    Tsallis entropy;   

DOI  :  10.3390/e13030595

来源: mdpi

【 摘 要 】

Kolmogorov complexity and Shannon entropy are conceptually different measures. However, for any recursive probability distribution, the expected value of Kolmogorov complexity equals its Shannon entropy, up to a constant. We study if a similar relationship holds for Rényi and Tsallis entropies of order α, showing that it only holds for , Tsallis and Rényi entropies converge if and only if α is greater than 1. We also establish the uniform continuity of these entropies.

【 授权许可】

CC BY   
© 2011 by the authors; licensee MDPI, Basel, Switzerland.