Knot selection in sparse Gaussian processes with a variational objective function

Statistical Analysis and Data Mining

article

Nathaniel Garton¹  Jarad Niemi¹  Alicia Carriquiry¹ 

[1] Department of Statistics, Iowa State University

关键词: knot selection;    machine learning;    nonparametric regression;    sparse Gaussian processes;    variational inference;   

DOI  :  10.1002/sam.11459

学科分类：社会科学、人文和艺术（综合）

来源: John Wiley & Sons, Inc.

【 摘 要 】

Sparse, knot-based Gaussian processes have enjoyed considerable success as scalable approximations of full Gaussian processes. Certain sparse models can be derived through specific variational approximations to the true posterior, and knots can be selected to minimize the Kullback-Leibler divergence between the approximate and true posterior. While this has been a successful approach, simultaneous optimization of knots can be slow due to the number of parameters being optimized. Furthermore, there have been few proposed methods for selecting the number of knots, and no experimental results exist in the literature. We propose using a one-at-a-time knot selection algorithm based on Bayesian optimization to select the number and locations of knots. We showcase the competitive performance of this method relative to optimization of knots simultaneously on three benchmark datasets, but at a fraction of the computational cost.

【 授权许可】

Unknown