Efficient Approximation of the Conditional Relative Entropy with Applications to Discriminative Learning of Bayesian Network Classifiers

Entropy

Alexandra M. Carvalho²  Pedro Adão¹ 

[1] Department of Computer Science, IST, University of Lisbon, Lisbon 1049-001, Portugal; E-Mail:;Department of Electrical Engineering, IST, University of Lisbon, Lisbon 1049-001, Portugal

关键词: conditional relative entropy;    approximation;    discriminative learning;    Bayesian network classifiers;   

DOI  :  10.3390/e15072716

来源: mdpi

【 摘 要 】

We propose a minimum variance unbiased approximation to the conditional relative entropy of the distribution induced by the observed frequency estimates, for multi-classification tasks. Such approximation is an extension of a decomposable scoring criterion, named approximate conditional log-likelihood (aCLL), primarily used for discriminative learning of augmented Bayesian network classifiers. Our contribution is twofold: (i) it addresses multi-classification tasks and not only binary-classification ones; and (ii) it covers broader stochastic assumptions than uniform distribution over the parameters. Specifically, we considered a Dirichlet distribution over the parameters, which was experimentally shown to be a very good approximation to CLL. In addition, for Bayesian network classifiers, a closed-form equation is found for the parameters that maximize the scoring criterion.

【 授权许可】

CC BY   
© 2013 by the authors; licensee MDPI, Basel, Switzerland.