Adversarial α-divergence minimization for Bayesian approximate inference

NEUROCOMPUTING

卷:471

Article

Rodriguez-Santana, Simon¹  Hernandez-Lobato, Daniel² 

[1] Inst Math Sci ICMAT CSIC, Campus Cantoblanco,C Nicolas Cabrera 13-15, Madrid 28049, Spain

[2] Univ Autonoma Madrid, Escuela Politecn Super, Campus Cantoblanco,C Franciso Tomas y Valiente 11, Madrid 28049, Spain

关键词: Bayesian neural networks;    Approximate inference;    Alpha divergences;    Adversarial variational Bayes;   

DOI  :  10.1016/j.neucom.2020.09.076

来源: Elsevier

【 摘 要 】

Neural networks are state-of-the-art models for machine learning problems. They are often trained via back-propagation to find a value of the weights that correctly predicts the observed data. Back-propagation has shown good performance in many applications, however, it cannot easily output an estimate of the uncertainty in the predictions made. Estimating this uncertainty is a critical aspect with important applications. One method to obtain this information consists in following a Bayesian approach to obtain a posterior distribution of the model parameters. This posterior distribution summarizes which parameter values are compatible with the observed data. However, the posterior is often intractable and has to be approximated. Several methods have been devised for this task. Here, we propose a general method for approximate Bayesian inference that is based on minimizing alpha-divergences, and that allows for flexible approximate distributions. We call this method adversarial alpha-divergence minimization (AADM). We have evaluated AADM in the context of Bayesian neural networks. Extensive experiments show that it may lead to better results in terms of the test log-likelihood, and sometimes in terms of the squared error, in regression problems. In classification problems, however, AADM gives competitive results. (C) 2020 Elsevier B.V. All rights reserved.

【 授权许可】

Free