L1-Penalized Ordinal Regression and Bayesian Variable Selection for Linear Models with Multiple Responses

Ordinal Regression;Gibbs sampling;Variable Selection;Latent variable;Probit model;Data augmentation;Statistics

Chang, Ya-Ting

University of Waterloo

关键词: Ordinal Regression;    Gibbs sampling;    Variable Selection;    Latent variable;    Probit model;    Data augmentation;    Statistics;   

Others  :  https://uwspace.uwaterloo.ca/bitstream/10012/9368/1/Chang_Ya-Ting.pdf

瑞士|英语

来源: UWSPACE Waterloo Institutional Repository

【 摘 要 】

Polychotomous ordinal response data are often analyzed by first introduce a latent continuous variable which can be modeled as an ordinary regression problem with the presence of covariates by using Markov chain Monte Carlo techniques. For variable selection purpose, we modified this approach by using the idea of Stochastic EM algorithm to infuse L-1 penalized regression in estimating the parameter of interest. This allows us to rank the variables in their order of significance based on posterior selection probabilities. We make comparisons with univariate Bayesian variable selection in the simulation and applied the proposed algorithm on data obtained from the MovieLens Project and the World Value Survey. Given the convenience of using Gibbs sampler to sample from the posterior distributions and choosing prior distributions based on the problems of our interest in Bayesian analyses, we extended the variable selection problem to consider multiple response data by allowing different sets of variables to be selected for different response variables through the infusion of additional information into the prior distribution of the selection variables. This contrasts with the usual approach to multiple response variable selection that selects a common set of variables for all of the response variables. In the simulation, we compared our proposed method against univariate Bayesian variable selection and it shows that the performance is improved after the infusion of relationship information.