Bayesian nonlinear regression for large p small n problems

JOURNAL OF MULTIVARIATE ANALYSIS

卷:108

Article

Chakraborty, Sounak¹  Ghosh, Malay²  Mallick, Bani K.³ 

[1] Univ Missouri, Dept Stat, Columbia, MO 65211 USA

[2] Univ Florida, Dept Stat, Gainesville, FL 32611 USA

[3] Texas A&M Univ, Dept Stat, College Stn, TX 77843 USA

关键词: Bayesian hierarchical model;    Empirical Bayes;    Gibbs sampling;    Markov chain Monte Carlo;    Metropolis-Hastings algorithm;    Near infrared spectroscopy;    Relevance vector machine;    Reproducing kernel Hilbert space;    Support vector machine;    Vapnik's epsilon-insensitive loss;   

DOI  :  10.1016/j.jmva.2012.01.015

来源: Elsevier

【 摘 要 】

Statistical modeling and inference problems with sample sizes substantially smaller than the number of available covariates are challenging. This is known as large p small n problem. Furthermore, the problem is more complicated when we have multiple correlated responses. We develop multivariate nonlinear regression models in this setup for accurate prediction. In this paper, we introduce a full Bayesian support vector regression model with Vapnik's epsilon-insensitive loss function, based on reproducing kernel Hilbert spaces (RKHS) under the multivariate correlated response setup. This provides a full probabilistic description of support vector machine (SVM) rather than an algorithm for fitting purposes. We have also introduced a multivariate version of the relevance vector Machine (RVM). Instead of the original treatment of the RVM relying on the use of type II maximum likelihood estimates of the hyper-parameters, we put a prior on the hyper-parameters and use Markov chain Monte Carlo technique for computation. We have also proposed an empirical Bayes method for our RVM and SVM. Our methods are illustrated with a prediction problem in the near-infrared (NIR) spectroscopy. A simulation study is also undertaken to check the prediction accuracy of our models. (c) 2012 Elsevier Inc. All rights reserved.

【 授权许可】

Free