Information-Theoretic Data Discarding for Dynamic Trees on Data Streams

Entropy

Robert B. Gramacy¹  Christoforos Anagnostopoulos² 

[1] Booth School of Business, The University of Chicago, 5807 South Woodlawn Avenue, Chicago, IL 60637, USA;Department of Mathematics, Imperial College London, South Kensington Campus, London SW7 2AZ, UK;

关键词: regression and classification trees;    dynamic trees;    streaming data;    massive data;    online learning;    active learning;   

DOI  :  10.3390/e15125510

来源: DOAJ

【 摘 要 】

Ubiquitous automated data collection at an unprecedented scale is making available streaming, real-time information flows in a wide variety of settings, transforming both science and industry. Learning algorithms deployed in such contexts often rely on single-pass inference, where the data history is never revisited. Learning may also need to be temporally adaptive to remain up-to-date against unforeseen changes in the data generating mechanism. Online Bayesian inference remains challenged by such transient, evolving data streams. Nonparametric modeling techniques can prove particularly ill-suited, as the complexity of the model is allowed to increase with the sample size. In this work, we take steps to overcome these challenges by porting information theoretic heuristics, such as exponential forgetting and active learning, into a fully Bayesian framework. We showcase our methods by augmenting a modern non-parametric modeling framework, dynamic trees, and illustrate its performance on a number of practical examples. The end product is a powerful streaming regression and classification tool, whose performance compares favorably to the state-of-the-art.

【 授权许可】

Unknown