Variational Inference over Nonstationary Data Streams for Exponential Family Models

Mathematics

卷:8

Darío Ramos-López¹  ThomasD. Nielsen²  Helge Langseth³  AndrésR. Masegosa⁴  Antonio Salmerón⁴ 

[1] Department of Applied Mathematics, Materials Science and Engineering, and Electronic Technology, Rey Juan Carlos University, 28933 Móstoles, Spain;

[2] Department of Computer Science, Aalborg University, 9220 Aalborg, Denmark;

[3] Department of Computer Science, Norwegian University of Science and Technology, 7491 Trondheim, Norway;

[4] Department of Mathematics and Center for Development and Transfer of Mathematical Research to Industry (CDTIME), University of Almería, 04120 Almería, Spain;

关键词: latent variable models;    nonstationary data streams;    concept drift;    variational inference;    power priors;    exponential forgetting;   

DOI  :  10.3390/math8111942

来源: DOAJ

【 摘 要 】

In many modern data analysis problems, the available data is not static but, instead, comes in a streaming fashion. Performing Bayesian inference on a data stream is challenging for several reasons. First, it requires continuous model updating and the ability to handle a posterior distribution conditioned on an unbounded data set. Secondly, the underlying data distribution may drift from one time step to another, and the classic i.i.d. (independent and identically distributed), or data exchangeability assumption does not hold anymore. In this paper, we present an approximate Bayesian inference approach using variational methods that addresses these issues for conjugate exponential family models with latent variables. Our proposal makes use of a novel scheme based on hierarchical priors to explicitly model temporal changes of the model parameters. We show how this approach induces an exponential forgetting mechanism with adaptive forgetting rates. The method is able to capture the smoothness of the concept drift, ranging from no drift to abrupt drift. The proposed variational inference scheme maintains the computational efficiency of variational methods over conjugate models, which is critical in streaming settings. The approach is validated on four different domains (energy, finance, geolocation, and text) using four real-world data sets.

【 授权许可】

Unknown