Parametric Bayesian Estimation of Differential Entropy and Relative Entropy

Entropy

Maya Gupta¹ 

[1] Department of Electrical Engineering, University of Washington, Seattle WA 98195-2500, USA

关键词: Kullback-Leibler;    relative entropy;    differential entropy;    Pareto;    Wishart;   

DOI  :  10.3390/e12040818

来源: mdpi

【 摘 要 】

Given iid samples drawn from a distribution with known parametric form, we propose the minimization of expected Bregman divergence to form Bayesian estimates of differential entropy and relative entropy, and derive such estimators for the uniform, Gaussian, Wishart, and inverse Wishart distributions. Additionally, formulas are given for a log gamma Bregman divergence and the differential entropy and relative entropy for the Wishart and inverse Wishart. The results, as always with Bayesian estimates, depend on the accuracy of the prior parameters, but example simulations show that the performance can be substantially improved compared to maximum likelihood or state-of-the-art nonparametric estimators.

【 授权许可】

CC BY   
© 2010 by the authors; licensee MDPI, Basel, Switzerland.