Prediction using step-wise L1, L2 regularization and feature selection for small data sets with large number of features

BMC Bioinformatics

Methodology Article

Tatsuya Akutsu¹  Mayumi Kamada¹  Ernst-Walter Knapp²  Ozgur Demir-Kavuk² 

[1] Bioinformatics Center, Institute for Chemical Research, Kyoto University, 611-0011, Uji, Kyoto, Japan;Institute of Chemistry and Biochemistry, Freie Universität Berlin, Fabeckstrasse 36A, 14195, Berlin, Germany;

关键词: Major Histocompatibility Complex;    Feature Selection;    Loss Function;    Prediction Performance;    Lasso;   

DOI  :  10.1186/1471-2105-12-412

 received in 2011-07-06, accepted in 2011-10-25,  发布年份 2011

来源: Springer

【 摘 要 】

BackgroundMachine learning methods are nowadays used for many biological prediction problems involving drugs, ligands or polypeptide segments of a protein. In order to build a prediction model a so called training data set of molecules with measured target properties is needed. For many such problems the size of the training data set is limited as measurements have to be performed in a wet lab. Furthermore, the considered problems are often complex, such that it is not clear which molecular descriptors (features) may be suitable to establish a strong correlation with the target property. In many applications all available descriptors are used. This can lead to difficult machine learning problems, when thousands of descriptors are considered and only few (e.g. below hundred) molecules are available for training.ResultsThe CoEPrA contest provides four data sets, which are typical for biological regression problems (few molecules in the training data set and thousands of descriptors). We applied the same two-step training procedure for all four regression tasks. In the first stage, we used optimized L1 regularization to select the most relevant features. Thus, the initial set of more than 6,000 features was reduced to about 50. In the second stage, we used only the selected features from the preceding stage applying a milder L2 regularization, which generally yielded further improvement of prediction performance. Our linear model employed a soft loss function which minimizes the influence of outliers.ConclusionsThe proposed two-step method showed good results on all four CoEPrA regression tasks. Thus, it may be useful for many other biological prediction problems where for training only a small number of molecules are available, which are described by thousands of descriptors.

【 授权许可】

Unknown   
© Demir-Kavuk et al; licensee BioMed Central Ltd. 2011. This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.