BioPreDyn-bench: a suite of benchmark problems for dynamic modelling in systems biology

BMC Systems Biology

Julio R Banga³  Johannes Jaeger¹  Pedro Mendes²  Eva Balsa-Canto³  Klaus Mauch⁴  Julio Saez-Rodriguez⁵  Anton Crombach¹  Damjan Cicin-Sain¹  Joachim Schmid⁴  Sophia Bongard⁴  Kieran Smallbone²  David Henriques⁵  Alejandro F Villaverde³ 

[1]Universitat Pompeu Fabra (UPF), Plaça de la Mercé, 10, Barcelona 08002, Spain

[2]School of Computer Science, Manchester Centre for Integrative Systems Biology, The University of Manchester, 131 Princess Street, Manchester M1 7DN, UK

[3]Bioprocess Engineering Group, IIM-CSIC, Eduardo Cabello 6, Vigo 36208, Spain

[4]Insilico Biotechnology AG, Meitnerstraße 8, Stuttgart 70563, Germany

[5]European Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome Campus, Hinxton CB10 1SD, Cambridge, UK

关键词: development;    Signal transduction;    Transcription;    Metabolism;    Large-scale;    Benchmarks;    Optimization;    Parameter estimation;    Model calibration;    Dynamic modelling;   

Others  :  1159569
DOI  :  10.1186/s12918-015-0144-4

 received in 2014-07-21, accepted in 2015-01-15,  发布年份 2015

【 摘 要 】

Background

Dynamic modelling is one of the cornerstones of systems biology. Many research efforts are currently being invested in the development and exploitation of large-scale kinetic models. The associated problems of parameter estimation (model calibration) and optimal experimental design are particularly challenging. The community has already developed many methods and software packages which aim to facilitate these tasks. However, there is a lack of suitable benchmark problems which allow a fair and systematic evaluation and comparison of these contributions.

Results

Here we present BioPreDyn-bench, a set of challenging parameter estimation problems which aspire to serve as reference test cases in this area. This set comprises six problems including medium and large-scale kinetic models of the bacterium E. coli, baker’s yeast S. cerevisiae, the vinegar fly D. melanogaster, Chinese Hamster Ovary cells, and a generic signal transduction network. The level of description includes metabolism, transcription, signal transduction, and development. For each problem we provide (i) a basic description and formulation, (ii) implementations ready-to-run in several formats, (iii) computational results obtained with specific solvers, (iv) a basic analysis and interpretation.

Conclusions

This suite of benchmark problems can be readily used to evaluate and compare parameter estimation methods. Further, it can also be used to build test problems for sensitivity and identifiability analysis, model reduction and optimal experimental design methods. The suite, including codes and documentation, can be freely downloaded from the BioPreDyn-bench website, https://sites.google.com/site/biopredynbenchmarks/ webcite.

【 授权许可】

   
2015 Villaverde et al.; licensee BioMed Central.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

【 参考文献 】

• [1]Link H, Christodoulou D, Sauer U: Advancing metabolic models with kinetic information. Curr Opin Biotechnol 2014, 29:8-14.
• [2]Almquist J, Cvijovic M, Hatzimanikatis V, Nielsen J, Jirstrand M: Kinetic models in industrial biotechnology–improving cell factory performance. Metab Eng. 2014, 24:38-60.
• [3]Song H-S, DeVilbiss F, Ramkrishna D: Modeling metabolic systems: the need for dynamics. Curr Opin Chem Eng. 2013, 2(4):373-82.
• [4]Jaeger J, Monk N: Bioattractors: Dynamical systems theory and the evolution of regulatory processes. J Physiol (Lond). 2014, 592:2267-81.
• [5]Villaverde AF, Banga JR: Reverse engineering and identification in systems biology: strategies, perspectives and challenges. J R Soc Interface. 2014, 11(91):20130505.
• [6]van Riel N.A.W: Dynamic modelling and analysis of biochemical networks: mechanism-based models and model-based experiments Brief Bioinform. 2006, 7(4):364-74.
• [7]Jaqaman K, Danuser G: Linking data to models: data regression. Nat Rev Mol Cell Biol. 2006, 7(11):813-19.
• [8]Banga J, Balsa-Canto E: Parameter estimation and optimal experimental design. Essays Biochem. 2008, 45:195.
• [9]Ashyraliyev M, Fomekong-Nanfack Y, Kaandorp JA, Blom JG: Systems biology: parameter estimation for biochemical models. FEBS J. 2008, 276(4):886-902.
• [10]Vanlier J, Tiemann C, Hilbers P: van Riel N: Parameter uncertainty in biochemical models described by ordinary differential equations. Math Biosci. 2013, 246(2):305-14.
• [11]Moles CG, Mendes P, Banga JR: Parameter estimation in biochemical pathways: a comparison of global optimization methods. Genome Res. 2003, 13(11):2467.
• [12]Banga JR: Optimization in computational systems biology. BMC Syst Biol. 2008, 2:47.
• [13]Mendes P, Sha W, Ye K: Artificial gene networks for objective comparison of analysis algorithms. Bioinformatics 2003, 19(suppl 2):122-29.
• [14]Kremling A, Fischer S, Gadkar K, Doyle FJ, Sauter T, Bullinger E, et al.: A benchmark for methods in reverse engineering and modeldiscrimination: problem formulation and solutions. Genome Res. 2004, 14(9):1773-85.
• [15]Camacho D, Vera Licona P, Mendes P, Laubenbacher R: Comparison of reverse-engineering methods using an in silico network. Ann N Y Acad Sci. 2007, 1115(1):73-89.
• [16]Gennemark P, Wedelin D: Benchmarks for identification of ordinary differential equations from time series data. Bioinformatics 2009, 25(6):780-86.
• [17]Haynes BC, Brent MR: Benchmarking regulatory network reconstruction with grendel. Bioinformatics 2009, 25(6):801-07.
• [18]Marbach D, Schaffter T, Mattiussi C, Floreano D: Generating realistic in silico gene networks for performance assessment of reverse engineering methods. J Comput Biol. 2009, 16(2):229-39.
• [19]Schaffter T, Marbach D, Floreano D: Genenetweaver: in silico benchmark generation and performance profiling of network inference methods. Bioinformatics 2011, 27(16):2263-70.
• [20]Meyer P, Cokelaer T, Chandran D, Kim KH, Loh P-R, Tucker G, et al.: Network topology and parameter estimation: from experimental design methods to gene regulatory network kinetics using a community based approach. BMC Syst Biol. 2014, 8(1):13.
• [21]Auger A, Hansen N, Schoenauer M: Benchmarking of continuous black box optimization algorithms. Evol Comput. 2012, 20(4):481.
• [22]Dolan ED, Moré JJ, Munson TS. Benchmarking optimization software with cops 3.0. Argonne National Laboratory Technical Report ANL/MCS-TM-273, 9700 South Cass Avenue, Argonne, Illinois 60439, USA. 2004.
• [23]Chassagnole C, Noisommit-Rizzi N, Schmid JW, Mauch K, Reuss M: Dynamic modeling of the central carbon metabolism of escherichia coli. Biotechnol Bioeng. 2002, 79(1):53-73.
• [24]Kotte O, Zaugg JB, Heinemann M: Bacterial adaptation through distributed sensing of metabolic fluxes. Mol Syst Biol. 2010, 6(1):355.
• [25]Smallbone K, Mendes P: Large-scale metabolic models: From reconstruction to differential equations. Ind Biotech. 2013, 9(4):179-84.
• [26]Villaverde AF, Bongard S, Schmid J, Müller D, Mauch K, Balsa-Canto E, et al.: High-confidence predictions in systems biology dynamic models. In Advances in Intelligent and Soft-Computing, vol. 294. Springer, Switzerland; 2014.
• [27]MacNamara A, Terfve C, Henriques D, Bernabé BP, Saez-Rodriguez J: State–time spectrum of signal transduction logic models. Phys Biol. 2012, 9(4):045003.
• [28]Jaeger J, Surkova S, Blagov M, Janssens H, Kosman D, Kozlov KN, et al.: Dynamic control of positional information in the early drosophila embryo. Nature 2004, 430(6997):368-71.
• [29]Crombach A, Wotton KR, Cicin-Sain D, Ashyraliyev M, Jaeger J: Efficient reverse-engineering of a developmental gene regulatory network. PLoS Comput Biol. 2012, 8(7):1002589.
• [30]Ashyraliyev M, Siggens K, Janssens H, Blom J, Akam M, Jaeger J: Gene circuit analysis of the terminal gap gene huckebein. PLoS Comput Biol. 2009, 5(10):1000548.
• [31]Krause F, Schulz M, Swainston N, Liebermeister W: Sustainable model building the role of standards and biological semantics. Methods Enzymol. 2011, 500:371-95.
• [32]Hucka M, Finney A, Sauro H. M, Bolouri H, Doyle JC, Kitano H, et al.: The systems biology markup language (sbml): a medium for representation and exchange of biochemical network models. Bioinformatics 2003, 19(4):524-31.
• [33]Balsa-Canto E, Banga JR: Amigo, a toolbox for advanced model identification in systems biology using global optimization. Bioinformatics 2011, 27(16):2311-3.
• [34]Hoops S, Sahle S, Gauges R, Lee C, Pahle J, Simus N, et al.: Copasi – a complex pathway simulator. Bioinformatics 2006, 22(24):3067-74.
• [35]Balsa-Canto E, Alonso A, Banga JR: An iterative identification procedure for dynamic modeling of biochemical networks. BMC Syst Biol. 2010, 4:11.
• [36]Egea JA, Martí R, Banga JR: An evolutionary method for complex-process optimization. Comput Oper Res 2010, 37(2):315-24.
• [37]Villaverde A, Egea J, Banga J: A cooperative strategy for parameter estimation in large scale systems biology models. BMC Syst Biol. 2012, 6(1):75.
• [38]Egea J, Henriques D, Cokelaer T, Villaverde A, MacNamara A, Danciu D. -P, et al.: Meigo: an open-source software suite based on metaheuristics for global optimization in systems biology and bioinformatics. BMC Bioinformatics 2014, 15:136.
• [39]Dolan E. D, Moré J. J: Benchmarking optimization software with performance profiles. Math Program, Ser A 2002, 91(2):201-13.
• [40]Walter E, Pronzato L: Identification of parametric models from experimental data. Communications and control engineering series. Springer, London, UK; 1997.
• [41]Gadkar KG, Gunawan R, Doyle FJ: Iterative approach to model identification of biological networks. BMC Bioinformatics 2005, 6(1):155.
• [42]Banga JR, Balsa-Canto E: Chiş O-T: Structural identifiability of systems biology models: a critical comparison of methods. PLoS ONE 2011, 6(11):27755.
• [43]Chiş O, Balsa-Canto E: Banga J.R: Genssi: a software toolbox for structural identifiability analysis of biological models. Bioinformatics 2011, 27(18):2610-1.
• [44]Becker K, Balsa-Canto E, Cicin-Sain D, Hoermann A, Janssens H, Banga JR, et al.: Reverse-engineering post-transcriptional regulation of gap genes in drosophila melanogaster. PLoS Comput Biol. 2013, 9(10):1003281.
• [45]Zak DE, Gonye GE, Schwaber JS, Doyle FJ: Importance of input perturbations and stochastic gene expression in the reverse engineering of genetic regulatory networks: insights from an identifiability analysis of an in silico network. Genome Res. 2003, 13(11):2396-405.
• [46]Yue H, Brown M, Knowles J, Wang H, Broomhead DS, Kell DB: Insights into the behaviour of systems biology models from dynamic sensitivity and identifiability analysis: a case study of an nf- κb signalling pathway. Mol Biosyst. 2006, 2(12):640-9.
• [47]Anguelova M, Cedersund G, Johansson M, Franzen C, Wennberg B: Conservation laws and unidentifiability of rate expressions in biochemical models. IET Syst Biol. 2007, 1(4):230-7.
• [48]Srinath S, Gunawan R: Parameter identifiability of power-law biochemical system models. J Biotechnol 2010, 149(3):132-40.
• [49]Szederkényi G, Banga JR, Alonso AA: Inference of complex biological networks: distinguishability issues and optimization-based solutions. BMC Syst Biol. 2011, 5(1):177.
• [50]Miao H, Xia X, Perelson AS, Wu H: On identifiability of nonlinear ode models and applications in viral dynamics. SIAM Rev. 2011, 53(1):3-39.
• [51]Jia G, Stephanopoulos G, Gunawan R: Incremental parameter estimation of kinetic metabolic network models. BMC Syst Biol. 2012, 6(1):142.
• [52]Cedersund G: Conclusions via unique predictions obtained despite unidentifiability–new definitions and a general method. FEBS J. 2012, 279(18):3513-27.
• [53]Berthoumieux S, Brilli M, Kahn D, De Jong H, Cinquemani E: On the identifiability of metabolic network models. J Math Biol. 2013, 67(6-7):1795-832.
• [54]DiStefano III J: Dynamic systems biology modeling and simulation. Academic Press, Waltham, MA, USA; 2014.
• [55]Sontag ED: For differential equations with r parameters, 2r+ 1 experiments are enough for identification. J Nonlinear Sci. 2002, 12(6):553-83.
• [56]Heavner BD, Smallbone K, Barker B, Mendes P, Walker LP: Yeast 5–an expanded reconstruction of the saccharomyces cerevisiae metabolic network. BMC Syst Biol. 2012, 6(1):55.
• [57]Smallbone K, Simeonidis E: Flux balance analysis: A geometric perspective. J Theor Biol 2009, 258(2):311-5.
• [58]Liebermeister W, Uhlendorf J, Klipp E: Modular rate laws for enzymatic reactions: thermodynamics, elasticities, and implementation. Bioinformatics 2010, 26(12):1528-34.
• [59]Li C, Donizelli M, Rodriguez N, Dharuri H, Endler L, Chelliah V, et al.: BioModels Database: an enhanced, curated and annotated resource for published quantitative kinetic models. BMC Syst Biol. 2010, 4:92.
• [60]Wurm FM: Production of recombinant protein therapeutics in cultivated mammalian cells. Nat Biotechnol. 2004, 22(11):1393-98.
• [61]Wittmann DM, Krumsiek J, Saez-Rodriguez J, Lauffenburger DA, Klamt S, Theis FJ: Transforming boolean models to continuous models: methodology and application to t-cell receptor signaling. BMC Syst Biol. 2009, 3(1):98.
• [62]Chaouiya C, Bérenguier D, Keating SM, Naldi A, Van Iersel M. P, Rodriguez N, et al.: Sbml qualitative models: a model representation format and infrastructure to foster interactions between qualitative modelling formalisms and tools. BMC Syst Biol. 2013, 7(1):135.
• [63]Mjolsness E, Sharp DH, Reinitz J: A connectionist model of development. J Theor Biol. 1991, 152(4):429-53.
• [64]Reinitz J, Sharp DH: Mechanism of eve stripe formation. Mech Dev. 1995, 49(1):133-58.
• [65]Rodríguez-Fernández M, Egea JA, Banga JR: Novel metaheuristic for parameter estimation in nonlinear dynamic biological systems. BMC Bioinformatics 2006, 7:483.