Hyperparameter learning in probabilistic prototype-based models

NEUROCOMPUTING

卷:73

Article

Schneider, Petra¹  Biehl, Michael¹  Hammer, Barbara² 

[1] Univ Groningen, Johann Bernoulli Inst Math & Comp Sci, NL-9700 AK Groningen, Netherlands

[2] Univ Bielefeld, Fac Technol, CITEC, D-33615 Bielefeld, Germany

关键词: Learning vector quantization;    Robust Soft LVQ;    Distance based classification;    Likelihood;    Cost function;    Hyperparameter;   

DOI  :  10.1016/j.neucom.2009.11.021

来源: Elsevier

【 摘 要 】

We present two approaches to extend Robust Soft Learning Vector Quantization (RSLVQ). This algorithm for nearest prototype classification is derived from an explicit cost function and follows the dynamics of a stochastic gradient ascent. The RSLVQ cost function is defined in terms of a likelihood ratio and involves a hyperparameter which is kept constant during training. We propose to adapt the hyperparameter in the training phase based on the gradient information. Besides, we propose to base the classifier's decision on the value of the likelihood ratio instead of using the distance based classification approach. Experiments on artificial and real life data show that the hyperparameter crucially influences the performance of RSLVQ. However, it is not possible to estimate the best value from the data prior to learning. We show that the proposed variant of RSLVQ is very robust with respect to the initial value of the hyperparameter. The classification approach based on the likelihood ratio turns out to be superior to distance based classification, if local hyperparameters are adapted for each prototype. (C) 2010 Elsevier B.V. All rights reserved.

【 授权许可】

Free