How informative is your kinetic model?: using resampling methods for model invalidation

BMC Systems Biology

Age K Smilde¹  Johan A Westerhuis¹  Huub CJ Hoefsloot¹  Dicle Hasdemir¹ 

[1]2Netherlands Metabolomics Centre, Leiden, The Netherlands

关键词: Forecast analysis;    Cross validation;    Resampling;    PCA;    SPCA;    Smooth principal components analysis;    Differential equations;    ODE;    Kinetic models;    Model invalidation;   

Others  :  866358
DOI  :  10.1186/1752-0509-8-61

 received in 2014-02-21, accepted in 2014-05-14,  发布年份 2014

【 摘 要 】

Background

Kinetic models can present mechanistic descriptions of molecular processes within a cell. They can be used to predict the dynamics of metabolite production, signal transduction or transcription of genes. Although there has been tremendous effort in constructing kinetic models for different biological systems, not much effort has been put into their validation. In this study, we introduce the concept of resampling methods for the analysis of kinetic models and present a statistical model invalidation approach.

Results

We based our invalidation approach on the evaluation of a kinetic model’s predictive power through cross validation and forecast analysis. As a reference point for this evaluation, we used the predictive power of an unsupervised data analysis method which does not make use of any biochemical knowledge, namely Smooth Principal Components Analysis (SPCA) on the same test sets. Through a simulations study, we showed that too simple mechanistic descriptions can be invalidated by using our SPCA-based comparative approach until high amount of noise exists in the experimental data. We also applied our approach on an eicosanoid production model developed for human and concluded that the model could not be invalidated using the available data despite its simplicity in the formulation of the reaction kinetics. Furthermore, we analysed the high osmolarity glycerol (HOG) pathway in yeast to question the validity of an existing model as another realistic demonstration of our method.

Conclusions

With this study, we have successfully presented the potential of two resampling methods, cross validation and forecast analysis in the analysis of kinetic models’ validity. Our approach is easy to grasp and to implement, applicable to any ordinary differential equation (ODE) type biological model and does not suffer from any computational difficulties which seems to be a common problem for approaches that have been proposed for similar purposes. Matlab files needed for invalidation using SPCA cross validation and our toy model in SBML format are provided at http://www.bdagroup.nl/content/Downloads/software/software.php webcite.

【 授权许可】

   
2014 Hasdemir et al.; licensee BioMed Central Ltd.

【 图 表 】

【 参考文献 】

• [1]Kotte O, Zaugg J, Heinemann M: Bacterial adaptation through distributed sensing of metabolic fluxes. Mol Syst Biol 2010, 6:355.
• [2]Gupta S, Maurya MR, Stephens DL, Dennis EA, Subramaniam S: An integrated model of eicosanoid metabolism and signaling based on lipidomics flux analysis. Biophys J 2009, 96(11):4542-51.
• [3]Teusink B, Passarge J, Reijenga CA, Esgalhado E, van der Weijden CC, Schepper M, Walsh MC, Bakker BM, van Dam K, Westerhoff HV, Snoep JL: Can yeast glycolysis be understood in terms of in vitro kinetics of the constituent enzymes? Testing biochemistry. Eur J Biochem 2000, 267(17):5313-5329.
• [4]du Preez FB, van Niekerk DD, Kooi B, Rohwer JM, Snoep JL: From steady-state to synchronized yeast glycolytic oscillations I: model construction. FEBS J 2012, 279(16):2810-2822.
• [5]Le Novère N, Bornstein B, Broicher A, Courtot M, Donizelli M, Dharuri H, Li L, Sauro H, Schilstra M, Shapiro B, Snoep JL, Hucka M: BioModels Database: a free, centralized database of curated, published, quantitative kinetic models of biochemical and cellular systems. Nucleic Acids Res 2006, 34(Database issue):D689-D691.
• [6]Timmer J, Müller TG, Swameye I, Sandra O, Klingmüller U: Modeling the nonlinear dynamics of cellular signal transduction. Int J Bifurcation Chaos 2004, 14(06):2069-2079.
• [7]du Preez FB, van Niekerk DD, Snoep JL: From steady-state to synchronized yeast glycolytic oscillations II: model validation. FEBS J 2012, 279(16):2823-2836.
• [8]Akaike H: A new look at the statistical model identification. Automatic Control IEEE Trans 1974, 19(6):716-723.
• [9]Turkheimer FE, Hinz R, Cunningham VJ: On the undecidability among kinetic models: from model selection to model averaging. J Cereb Blood Flow Metab 2003, 23(4):490-498.
• [10]Link H, Kochanowski K, Sauer U: Systematic identification of allosteric protein-metabolite interactions that control enzyme activity in vivo. Nat Biotech 2013, 31(4):357-361.
• [11]Flotmann M, Schaber J, Hoops S, Klipp E, Mendes P: ModelMage: a tool for automatic model generation, selection and management. Genome Inform 2008, 20:52-63.
• [12]Haunschild MD, Freisleben B, Takors R, Wiechert W: Investigating the dynamic behavior of biochemical networks using model families. Bioinformatics 2005, 21(8):1617-1625.
• [13]Cedersund G, Roll J: Systems biology: model based evaluation and comparison of potential explanations for given biological data. FEBS J 2009, 276(4):903-922.
• [14]Vyshemirsky V, Girolami M: Bayesian ranking of biochemical system models. Bioinformatics (Oxford, England) 2008, 24(6):833-839.
• [15]Toni T, Stumpf MPH: Simulation-based model selection for dynamical systems in systems and population biology. Bioinformatics 2010, 26:104-110.
• [16]Milias-Argeitis A, Porreca R, Summers S, Lygeros J: Bayesian model selection for the yeast GATA-factor network: a comparison of computational approaches. Decision and Control (CDC), 2010 49th IEEE Conference on 2010, 3379-3384.
• [17]Bates DG, Cosentino C: Validation and invalidation of systems biology models using robustness analysis. IET Syst Biol 2011, 5(4):229-44.
• [18]Hafner M, Koeppl H, Hasler M, Wagner A: ‘Glocal’ robustness analysis and model discrimination for circadian oscillators. PLoS Comput Biol 2009, 5(10):e1000534.
• [19]Anderson J, Papachristodoulou A: On validation and invalidation of biological models. BMC Bioinformatics 2009, 10:1-13.
• [20]Mendes P, Camacho D, de la Fuente A: Modelling and simulation for metabolomics data analysis. Biochem Soc Trans 2005, 33(Pt 6):1427-1429.
• [21]Buczynski MW, Dumlao DS, Dennis EA: Thematic review series: proteomics. An integrated omics analysis of eicosanoid biology. J Lipid Res 2009, 50(6):1015-1038.
• [22]Yang K, Ma W, Liang H, Ouyang Q, Tang C, Lai L: Dynamic simulations on the arachidonic acid metabolic network. PLoS Comput Biol 2007, 3(3):e55.
• [23]Hohmann S: Osmotic stress signaling and osmoadaptation in yeasts. Microbiol Mol Biol Rev 2002, 66(2):300-372.
• [24]Gustin MC, Albertyn J, Alexander M, Davenport K: MAP Kinase Pathways in the YeastSaccharomyces cerevisiae. Microbiol Mol Biol Revs 1998, 62(4):1264-1300.
• [25]Posas F, Wurgler-Murphy SM, Maeda T, Witten EA, Thai TC, Saito H: Yeast HOG1 MAP kinase cascade is regulated by a multistep phosphorelay mechanism in the SLN1–YPD1–SSK1 “two-component” osmosensor. Cell 1996, 86(6):865-875.
• [26]Schaber J, Baltanas R, Bush A, Klipp E, Colman-Lerner A: Modelling reveals novel roles of two parallel signalling pathways and homeostatic feedbacks in yeast. Mol Syst Biol 2012, 8:622.
• [27]Macia J, Regot S, Peeters T, Conde N, Sole R, Posas F: Dynamic signaling in the Hog1 MAPK pathway relies on high basal signal transduction. Sci Signal 2009, 2(63):ra13.
• [28]Bro R, Kjeldahl K, Smilde AK, Kiers HaL: Cross-validation of component models: a critical look at current methods. Anal Bioanal Chem 2008, 390(5):1241-1251.
• [29]Alpaydin E: Introduction to Machine Learning. Cambridge: MIT press; 2004.
• [30]Box GEP, Jenkins GM: Time Series Analysis: Forecasting and Control. Holden Day: San Francisco; 1976.
• [31]Coleman TF, Li Y: An interior trust region approach for nonlinear minimization subject to bounds. SIAM J Optimization 1996, 6(2):418-445.
• [32]Verouden MP: Fusing Prior Knowledge with microbial metabolomics. PhD thesis, University of Amsterdam 2012.
• [33]Jolliffe I: Principal Component Analysis. New York: Springer-Verlag; 2002.
• [34]Jansen JJ, Hoefsloot HCJ, Boelens HFM, van der Greef J, Smilde AK: Analysis of longitudinal metabolomics data. Bioinformatics (Oxford, England) 2004, 20(15):2438-2446.
• [35]Kiers HaL: Weighted least squares fitting using ordinary least squares algorithms. Psychometrika 1997, 62(2):251-266.
• [36]Josse J, Husson F: Handling missing values in exploratory multivariate data analysis methods. J de la Société Française de Stat 2012, 153(2):79-99.
• [37]Smit S, van Breemen MJ, Hoefsloot HC, Smilde AK, Aerts JM, de Koster CG: Assessing the statistical validity of proteomics based biomarkers. Anal Chimica Acta 2007, 592(2):210-217.
• [38]Toni T, Welch D, Strelkowa N, Ipsen A, Stumpf MP: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J R Soc Interface 2009, 6(31):187-202.
• [39]Moles CG, Mendes P, Banga JR: Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res 2003, 13(11):2467-2474.
• [40]Mendes P, Kell D: Non-linear optimization of biochemical pathways: applications to metabolic engineering and parameter estimation. Bioinformatics 1998, 14(10):869-883.
• [41]Kuepfer L, Peter M, Sauer U, Stelling J: Ensemble modeling for analysis of cell signaling dynamics. Nat Biotechnol 2007, 25(9):1001-1006.