Optimal experiment design for model selection in biochemical networks

BMC Systems Biology

Natal AW van Riel¹  Peter AJ Hilbers¹  Christian A Tiemann¹  Joep Vanlier¹ 

[1] Netherlands Consortium for Systems Biology, University of Amsterdam, Amsterdam, 1098 XH, The Netherlands

关键词: Uncertainty;    Bayes factor;    Inference;    Model selection;   

Others  :  1141341
DOI  :  10.1186/1752-0509-8-20

 received in 2013-04-11, accepted in 2014-02-13,  发布年份 2014

【 摘 要 】

Background

Mathematical modeling is often used to formalize hypotheses on how a biochemical network operates by discriminating between competing models. Bayesian model selection offers a way to determine the amount of evidence that data provides to support one model over the other while favoring simple models. In practice, the amount of experimental data is often insufficient to make a clear distinction between competing models. Often one would like to perform a new experiment which would discriminate between competing hypotheses.

Results

We developed a novel method to perform Optimal Experiment Design to predict which experiments would most effectively allow model selection. A Bayesian approach is applied to infer model parameter distributions. These distributions are sampled and used to simulate from multivariate predictive densities. The method is based on a k-Nearest Neighbor estimate of the Jensen Shannon divergence between the multivariate predictive densities of competing models.

Conclusions

We show that the method successfully uses predictive differences to enable model selection by applying it to several test cases. Because the design criterion is based on predictive distributions, which can be computed for a wide range of model quantities, the approach is very flexible. The method reveals specific combinations of experiments which improve discriminability even in cases where data is scarce. The proposed approach can be used in conjunction with existing Bayesian methodologies where (approximate) posteriors have been determined, making use of relations that exist within the inferred posteriors.

【 授权许可】

   
2014 Vanlier et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

【 参考文献 】

• [1]Tiemann C, Vanlier J, Hilbers P, van Riel N: Parameter adaptations during phenotype transitions in progressive diseases. BMC Syst Biol 2011, 5:174. BioMed Central Full Text
• [2]van Riel NA, Tiemann CA, Vanlier J, Hilbers PA: Applications of analysis of dynamic adaptations in parameter trajectories. Interface Focus 2013, 3(2):20120084.
• [3]Schmitz J, Van Riel N, Nicolay K, Hilbers P, Jeneson J: Silencing of glycolysis in muscle: experimental observation and numerical analysis. Exp Physiol 2010, 95(2):380-397.
• [4]Schilling M, Maiwald T, Hengl S, Winter D, Kreutz C, Kolch W, Lehmann W, Timmer J, Klingmüller U: Theoretical and experimental analysis links isoform-specific ERK signalling to cell fate decisions. Mol Syst Biol 2009, 5:334.
• [5]Borisov N, Aksamitiene E, Kiyatkin A, Legewie S, Berkhout J, Maiwald T, Kaimachnikov N, Timmer J, Hoek J, Kholodenko B: Systems-level interactions between insulin–EGF networks amplify mitogenic signaling. Mol Syst Biol 2009, 5:256.
• [6]Cedersund G, Roll J, Ulfhielm E, Danielsson A, Tidefelt H, Strålfors P: Model-based hypothesis testing of key mechanisms in initial phase of insulin signaling. PLoS Comput Biol 2008, 4(6):799-806.
• [7]Koschorreck M, Gilles E: Mathematical modeling and analysis of insulin clearance in vivo. BMC Syst Biol 2008, 2:43. BioMed Central Full Text
• [8]Schoeberl B, Eichler-Jonsson C, Gilles E, Müller G: Computational modeling of the dynamics of the MAP kinase cascade activated by surface and internalized EGF receptors. Nat Biotechnol 2002, 20(4):370-375.
• [9]Jeneson J, Westerhoff H, Kushmerick M: A metabolic control analysis of kinetic controls in ATP free energy metabolism in contracting skeletal muscle. Am J Physiol-Cell Physiol 2000, 279(3):C813-C832.
• [10]Wu F, Jeneson J, Beard D: Oxidative ATP synthesis in skeletal muscle is controlled by substrate feedback. Am J Physiol-Cell Physiol 2007, 292:C115-C124.
• [11]Groenendaal W, Schmidt K, von Basum G, van Riel N, Hilbers P: Modeling glucose and water dynamics in human skin. Diabetes Technol Ther 2008, 10(4):283-293.
• [12]Vanlier J, Wu F, Qi F, Vinnakota K, Han Y, Dash R, Yang F, Beard D: BISEN: Biochemical simulation environment. Bioinformatics 2009, 25(6):836-837.
• [13]Vanlier J, Tiemann C, Hilbers P, van Riel N: Parameter uncertainty in biochemical models described by ordinary differential equations. Math Biosci 2013, 246(2):305-314.
• [14]Klinke D: An empirical Bayesian approach for model-based inference of cellular signaling networks. BMC Bioinformatics 2009, 10:371. BioMed Central Full Text
• [15]Taylor H, Barnes C, Huvet M, Bugeon L, Thorne T, Lamb J, Dallman M, Stumpf M, et al.: Calibrating spatio-temporal models of leukocyte dynamics against in vivo live-imaging data using approximate Bayesian computation. Integr Biol 2012, 4(3):335-345.
• [16]Raue A, Kreutz C, Maiwald T, Bachmann J, Schilling M, Klingmüller U, Timmer J: Structural and practical identifiability analysis of partially observed dynamical models by exploiting the profile likelihood. Bioinformatics 2009, 25(15):1923.
• [17]Hasenauer J, Waldherr S, Wagner K, Allgower F: Parameter identification, experimental design and model falsification for biological network models using semidefinite programming. Syst Biol IET 2010, 4(2):119-130.
• [18]Brännmark C, Palmér R, Glad S, Cedersund G, Strålfors P: Mass and information feedbacks through receptor endocytosis govern insulin signaling as revealed using a parameter-free modeling framework. J Biol Chem 2010, 285(26):20171.
• [19]Girolami M, Calderhead B: Riemann manifold langevin and hamiltonian monte carlo methods. J R Stat Soci: Series B (Stat Methodol) 2011, 73(2):123-214.
• [20]Cedersund G, Roll J: Systems biology: model based evaluation and comparison of potential explanations for given biological data. FEBS J 2009, 276(4):903-922.
• [21]Müller T, Faller D, Timmer J, Swameye I, Sandra O, Klingmüller U: Tests for cycling in a signalling pathway. J R Stat Soc: Series C (Appl Stat) 2004, 53(4):557-568.
• [22]Calderhead B, Girolami M: Statistical analysis of nonlinear dynamical systems using differential geometric sampling methods. Interface Focus 2011, 1(6):821-835.
• [23]Tegnér J, Compte A, Auffray C, An G, Cedersund G, Clermont G, Gutkin B, Oltvai Z, Stephan K, Thomas R, et al.: Computational disease modeling–fact or fiction? BMC Syst Biol 2009, 3:56. BioMed Central Full Text
• [24]Skanda D, Lebiedz D: An optimal experimental design approach to model discrimination in dynamic biochemical systems. Bioinformatics 2010, 26(7):939.
• [25]Steiert B, Raue A, Timmer J, Kreutz C: Experimental Design for Parameter Estimation of Gene Regulatory Networks. PloS one 2012, 7(7):e40052.
• [26]Casey F, Baird D, Feng Q, Gutenkunst R, Waterfall J, Myers C, Brown K, Cerione R, Sethna J: Optimal experimental design in an epidermal growth factor receptor signalling and down-regulation model. Syst Biol IET 2007, 1(3):190-202.
• [27]Kreutz C, Raue A, Timmer J: Likelihood based observability analysis and confidence intervals for predictions of dynamic models. BMC Syst Biol 2012, 6:120. BioMed Central Full Text
• [28]Vanlier J, Tiemann C, Hilbers P, van Riel N: An integrated strategy for prediction uncertainty analysis. Bioinformatics 2012, 28(8):1130-1135.
• [29]Liepe J, Filippi S, Komorowski M, Stumpf MP: Maximizing the Information Content of Experiments in Systems Biology. PLOS Comput Biol 2013, 9:e1002888.
• [30]Vanlier J, Tiemann C, Hilbers P, van Riel N: A Bayesian approach to targeted experiment design. Bioinformatics 2012, 28(8):1136-1142.
• [31]King RD, Whelan KE, Jones FM, Reiser PG, Bryant CH, Muggleton SH, Kell DB, Oliver SG: Functional genomic hypothesis generation and experimentation by a robot scientist. Nature 2004, 427(6971):247-252.
• [32]MacKay DJ: Information-based objective functions for active data selection. Neural Comput 1992, 4(4):590-604.
• [33]Daunizeau J, Preuschoff K, Friston K, Stephan K: Optimizing experimental design for comparing models of brain function. PLoS Comput Biol 2011, 7(11):e1002280.
• [34]Hug S, Raue A, Hasenauer J, Bachmann J, Klingmüller U, Timmer J, Theis F: High-dimensional Bayesian parameter estimation: case study for a model of JAK2/STAT5 signaling. Math Biosci 2013, 246(2):293-304.
• [35]Finley S, Gupta D, Cheng N, Klinke D: Inferring relevant control mechanisms for interleukin-12 signaling in naïve CD4+; T cells. Immunol Cell Biol 2010, 89:100-110.
• [36]Konukoglu E, Relan J, Cilingir U, Menze B, Chinchapatnam P, Jadidi A, Cochet H, Hocini M, Delingette H, Jaïs P, et al.: Efficient probabilistic model personalization integrating uncertainty on data and parameters: Application to eikonal-diffusion models in cardiac electrophysiology. Prog Biophys Mol Biol 2011, 107:134-146.
• [37]Xu T, Vyshemirsky V, Gormand A, von Kriegsheim A, Girolami M, Baillie G, Ketley D, Dunlop A, Milligan G, Houslay M, et al.: Inferring signaling pathway topologies from multiple perturbation measurements of specific biochemical species. Sci Signal 2010, 3(113):ra20.
• [38]Kalita MK, Sargsyan K, Tian B, Paulucci-Holthauzen A, Najm HN, Debusschere BJ, Brasier AR: Sources of cell-to-cell variability in canonical nuclear factor-κ B (NF-κ B) signaling pathway inferred from single cell dynamic images. J Biol Chem 2011, 286(43):37741-37757.
• [39]Toni T, Stumpf M: Simulation-based model selection for dynamical systems in systems and population biology. Bioinformatics 2010, 26:104-110.
• [40]Vyshemirsky V, Girolami M: Bayesian ranking of biochemical system models. Bioinformatics 2008, 24(6):833-839.
• [41]Schmidl D, Hug S, Li WB, Greiter MB, Theis FJ: Bayesian model selection validates a biokinetic model for zirconium processing in humans. BMC Syst Biol 2012, 6:95. BioMed Central Full Text
• [42]Mélykúti B, August E, Papachristodoulou A, El-Samad H: Discriminating between rival biochemical network models: three approaches to optimal experiment design. BMC Syst Biol 2010, 4:38. BioMed Central Full Text
• [43]Flassig R, Sundmacher K: Optimal design of stimulus experiments for robust discrimination of biochemical reaction networks. Bioinformatics 2012, 28(23):3089-3096.
• [44]Endres DM, Schindelin JE: A new metric for probability distributions. Inf Theory IEEE Trans 2003, 49(7):1858-1860.
• [45]Kreutz C, Rodriguez M, Maiwald T, Seidl M, Blum H, Mohr L, Timmer J: An error model for protein quantification. Bioinformatics 2007, 23(20):2747.
• [46]Raue A, Kreutz C, Theis F, Timmer J: Joining forces of Bayesian and frequentist methodology: A study for inference in the presence of non-identifiability. Phil Trans Roy Soc A 2012, 371(1984):20110544.
• [47]Geyer C: Practical markov chain monte carlo. Stat Sci 1992, 7(4):473-483.
• [48]Toni T, Welch D, Strelkowa N, Ipsen A, Stumpf M: Approximate Bayesian computation scheme for parameter inference and model selection in dynamical systems. J R Soc Interface 2009, 6(31):187-202.
• [49]Burnham KP, Anderson DR: Model selection and multi-model inference: a practical information-theoretic approach. New York: Springer; 2002.
• [50]Penny WD, Stephan K, Mechelli A, Friston K: Comparing dynamic causal models. NeuroImage 2004, 22(3):1157-1172.
• [51]Good IJ: Weight of evidence: a brief survey. Bayesian Stat 1985, 2:249-269.
• [52]Calderhead B, Girolami M: Estimating Bayes factors via thermodynamic integration and population MCMC. Comput Stat Data Anal 2009, 53(12):4028-4045.
• [53]Lin J: Divergence measures based on the Shannon entropy. Inf Theory IEEE Trans 1991, 37:145-151.
• [54]Härdle W, Werwatz A, Müller M, Sperlich S: Introduction. Nonparametric Semiparametric Models. New York: Springer; 2004.
• [55]Kraskov A, Stögbauer H, Grassberger P: Estimating mutual information. Phys Rev E 2004, 69(6):066138.
• [56]Budka M, Gabrys B, Musial K: On accuracy of PDF divergence estimators and their applicability to representative data sampling. Entropy 2011, 13(7):1229-1266.
• [57]Boltz S, Debreuve E, Barlaud M: High-dimensional statistical distance for region-of-interest tracking: Application to combining a soft geometric constraint with radiometry. In IEEE Conference on Computer Vision and Pattern Recognition, 2007. CVPR’07. Minneapolis, Minnesota, USA: IEEE Computer Society; 2007:1-8.
• [58]Ogata H, Goto S, Sato K, Fujibuchi W, Bono H, Kanehisa M: KEGG: Kyoto encyclopedia of genes and genomes. Nucleic Acids Res 1999, 27:29-34.
• [59]Bevan P: Insulin signalling. J Cell Sci 2001, 114(8):1429-1430.
• [60]Friedman JH, Bentley JL, Finkel RA: An algorithm for finding best matches in logarithmic expected time. ACM Trans Math Softw (TOMS) 1977, 3(3):209-226.
• [61]Trotta R: Forecasting the Bayes factor of a future observation. Mon Notices R Astronomical Soc 2007, 378(3):819-824.
• [62]Calderhead B, Girolami M: Statistical analysis of nonlinear dynamical systems using differential geometric sampling methods. Interface Focus 2011, 1(6):821-835.
• [63]Calderhead B, Girolami M, Lawrence N: Accelerating Bayesian inference over nonlinear differential equations with Gaussian processes. Adv Neural Inf Process Syst 2009, 21:217-224.
• [64]Liepe J, Barnes C, Cule E, Erguler K, Kirk P, Toni T, Stumpf M: ABC-SysBio approximate Bayesian computation in Python with GPU support. Bioinformatics 2010, 26(14):1797.
• [65]Arefin A, Riveros C, Berretta R, Moscato P: GPU-FS-kNN: A Software Tool for Fast and Scalable kNN Computation Using GPUs. PloS one 2012, 7(8):e44000.
• [66]Garcia V, Debreuve E, Barlaud M: Fast k nearest neighbor search using GPU. In CVPR Workshop on Computer Vision on GPU. Anchorage, Alaska, USA: IEEE Computer Society; 2008.