Nonparametric Estimation of Distributional Functionals and Applications.

divergence estimation;mutual information estimation;entropy estimation;ensemble estimation;active region;high frequency oscillation;Electrical Engineering;Engineering;Electrical Engineering: Systems

Moon, Kevin R.Nadakuditi, Rajesh Rao ;

University of Michigan

关键词: divergence estimation;    mutual information estimation;    entropy estimation;    ensemble estimation;    active region;    high frequency oscillation;    Electrical Engineering;    Engineering;    Electrical Engineering: Systems;   

Others  :  https://deepblue.lib.umich.edu/bitstream/handle/2027.42/133394/krmoon_1.pdf?sequence=1&isAllowed=y

瑞士|英语

来源: The Illinois Digital Environment for Access to Learning and Scholarship

【 摘 要 】

Distributional functionals are integrals of functionals of probability densities and include functionals such as information divergence, mutual information, and entropy. Distributional functionals have many applications in the fields of information theory, statistics, signal processing, and machine learning. Many existing nonparametric distributional functional estimators have either unknown convergence rates or are difficult to implement. In this thesis, we consider the problem of nonparametrically estimating functionals of distributions when only a finite population of independent and identically distributed samples are available from each of the unknown, smooth, d-dimensional distributions. We derive mean squared error (MSE) convergence rates for leave-one-out kernel density plug-in estimators and k-nearest neighbor estimators of these functionals. We then extend the theory of optimally weighted ensemble estimation to obtain estimators that achieve the parametric MSE convergence rate when the densities are sufficiently smooth. These estimators are simple to implement anddo not require knowledge of the densities’ support set, in contrast with many competing estimators. The asymptotic distribution of these estimators is also derived. The utility of these estimators is demonstrated through their application to sunspot image data and neural data measured from epilepsy patients. Sunspot images are clustered by estimating the divergence between the underlying probability distributions of image pixel patches. The problem of overfitting is also addressed in both applications by performing dimensionality reduction via intrinsic dimension estimation and by benchmarking classification via Bayes error estimation