Learning curves for drug response prediction in cancer cell lines

BMC Bioinformatics

Michael Fonstein¹  Rick L. Stevens²  Songhao Jiang³  James H. Doroshow⁴  Austin Clyde⁵  Yitan Zhu⁶  Hyunseung Yoo⁶  Alexander Partin⁶  Fangfang Xia⁶  Maulik Shukla⁶  Yvonne A. Evrard⁷  Thomas Brettin⁸ 

[1] Biosciences Division, Argonne National Laboratory, Lemont, IL, USA;Computing, Environment and Life Sciences, Argonne National Laboratory, Lemont, IL, USA;Department of Computer Science, University of Chicago, Chicago, IL, USA;Department of Computer Science, University of Chicago, Chicago, IL, USA;Division of Cancer Therapeutics and Diagnosis, National Cancer Institute, Bethesda, MD, USA;Division of Data Science and Learning, Argonne National Laboratory, Lemont, IL, USA;Department of Computer Science, University of Chicago, Chicago, IL, USA;Division of Data Science and Learning, Argonne National Laboratory, Lemont, IL, USA;University of Chicago Consortium for Advanced Science and Engineering, University of Chicago, Chicago, IL, USA;Frederick National Laboratory for Cancer Research, Leidos Biomedical Research Inc., Frederick, MD, USA;University of Chicago Consortium for Advanced Science and Engineering, University of Chicago, Chicago, IL, USA;Computing, Environment and Life Sciences, Argonne National Laboratory, Lemont, IL, USA;

关键词: Learning curve;    Power law;    Drug response prediction;    Cell line;    Deep learning;    Machine learning;   

DOI  :  10.1186/s12859-021-04163-y

来源: Springer

【 摘 要 】

BackgroundMotivated by the size and availability of cell line drug sensitivity data, researchers have been developing machine learning (ML) models for predicting drug response to advance cancer treatment. As drug sensitivity studies continue generating drug response data, a common question is whether the generalization performance of existing prediction models can be further improved with more training data.MethodsWe utilize empirical learning curves for evaluating and comparing the data scaling properties of two neural networks (NNs) and two gradient boosting decision tree (GBDT) models trained on four cell line drug screening datasets. The learning curves are accurately fitted to a power law model, providing a framework for assessing the data scaling behavior of these models.ResultsThe curves demonstrate that no single model dominates in terms of prediction performance across all datasets and training sizes, thus suggesting that the actual shape of these curves depends on the unique pair of an ML model and a dataset. The multi-input NN (mNN), in which gene expressions of cancer cells and molecular drug descriptors are input into separate subnetworks, outperforms a single-input NN (sNN), where the cell and drug features are concatenated for the input layer. In contrast, a GBDT with hyperparameter tuning exhibits superior performance as compared with both NNs at the lower range of training set sizes for two of the tested datasets, whereas the mNN consistently performs better at the higher range of training sizes. Moreover, the trajectory of the curves suggests that increasing the sample size is expected to further improve prediction scores of both NNs. These observations demonstrate the benefit of using learning curves to evaluate prediction models, providing a broader perspective on the overall data scaling characteristics.ConclusionsA fitted power law learning curve provides a forward-looking metric for analyzing prediction performance and can serve as a co-design tool to guide experimental biologists and computational scientists in the design of future experiments in prospective research studies.

【 授权许可】

CC BY