Multiclass Latent Locally Linear Support Vector Machines

5th Asian Conference on Machine Learning

数学科学;计算机科学

Marco Fornoni [e-mail] ; Idiap Research Institute ; Martigny ; Switzerland

PID  :  123093

来源: CEUR

【 摘 要 】

Kernelized Support Vector Machines (SVM) have gained the status of offtheshelf clas sifiers, able to deliver state of the art performance on almost any problem. Still, their practical use is constrained by their computational and memory complexity, which grows superlinearly with the number of training samples. In order to retain the low training and testing complexity of linear classifiers and the flexibility of non linear ones, a growing, promising alternative is represented by methods that learn nonlinear classifiers through local combinations of linear ones. In this paper we propose a new multi class local classifier, based on a latent SVM formulation. The proposed classifier makes use of a set of linear models that are linearly combined using sample and class specific weights. Thanks to the latent formulation, the combination coefficients are modeled as latent variables. We allow soft combinations and we provide a closedform solution for their estimation, resulting in an efficient prediction rule. This novel formulation allows to learn in a principled way the sample specific weights and the linear classifiers, in a unique optimization problem, using a CCCP optimization procedure. Extensive experiments on ten standard UCI machine learning datasets, one large binary dataset, three character and digit recognition databases, and a visual place