Redundancy of Exchangeable Estimators

Entropy

Narayana P. Santhanam¹  Anand D. Sarwate² 

[1] Department of Electrical Engineering, University of Hawaii at Manoa, 2540 Dole Street, Honolulu, HI 96822, USA;Department of Electrical and Computer Engineering, Rutgers, The State University of New Jersey, 94 Brett Road, Piscataway, NJ 08854, USA

关键词: exchangeability;    random exchangeable partitions;    Chinese restaurant process;    Pitman-Yor process;    strong and weak universal compression;   

DOI  :  10.3390/e16105339

来源: mdpi

【 摘 要 】

Exchangeable random partition processes are the basis for Bayesian approaches to statistical inference in large alphabet settings. On the other hand, the notion of the pattern of a sequence provides an information-theoretic framework for data compression in large alphabet scenarios. Because data compression and parameter estimation are intimately related, we study the redundancy of Bayes estimators coming from Poisson–Dirichlet priors (or “Chinese restaurant processes”) and the Pitman–Yor prior. This provides an understanding of these estimators in the setting of unknown discrete alphabets from the perspective of universal compression. In particular, we identify relations between alphabet sizes and sample sizes where the redundancy is small, thereby characterizing useful regimes for these estimators.

【 授权许可】

CC BY   
© 2014 by the authors; licensee MDPI, Basel, Switzerland