Improving basic and translational science by accounting for litter-to-litter variation in animal models

BMC Neuroscience

Laurent Essioux¹  Stanley E Lazic² 

[1] , Bioinformatics and Exploratory Data Analysis, F. Hoffmann-La Roche, Basel, Switzerland;In Silico Lead Discovery, Novartis Institutes for Biomedical Research, Switzerland

关键词: Valproic acid;    Nested model;    Multiparous;    Mixed-effects model;    Litter-effects;    Experimental design;    Autism;   

Others  :  1140350
DOI  :  10.1186/1471-2202-14-37

 received in 2012-10-10, accepted in 2013-03-08,  发布年份 2013

【 摘 要 】

Background

Animals from the same litter are often more alike compared with animals from different litters. This litter-to-litter variation, or “litter effects”, can influence the results in addition to the experimental factors of interest. Furthermore, sometimes an experimental treatment can only be applied to whole litters rather than to individual offspring. An example is the valproic acid (VPA) model of autism, where VPA is administered to pregnant females thereby inducing the disease phenotype in the offspring. With this type of experiment the sample size is the number of litters and not the total number of offspring. If such experiments are not appropriately designed and analysed, the results can be severely biased as well as extremely underpowered.

Results

A review of the VPA literature showed that only 9% (3/34) of studies correctly determined that the experimental unit (n) was the litter and therefore made valid statistical inferences. In addition, litter effects accounted for up to 61% (p <0.001) of the variation in behavioural outcomes, which was larger than the treatment effects. In addition, few studies reported using randomisation (12%) or blinding (18%), and none indicated that a sample size calculation or power analysis had been conducted.

Conclusions

Litter effects are common, large, and ignoring them can make replication of findings difficult and can contribute to the low rate of translating preclinical in vivo studies into successful therapies. Only a minority of studies reported using rigorous experimental methods, which is consistent with much of the preclinical in vivo literature.

【 授权许可】

   
2013 Lazic and Essioux; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 7.

Figure 4.

【 参考文献 】

• [1]Geerts H: Of mice and men: bridging the translational disconnect in CNS drug discovery. CNS Drugs 2009, 23(11):915-926. [http://dx.doi.org/10.2165/11310890-000000000-00000 webcite]
• [2]Haseman JK, Hogan MD: Selection of the experimental unit in teratology studies. Teratol 1975, 12(2):165-171.
• [3]Hughes CW: Outcome of early experience studies as affected by between-litter variance. J Nutr 1979, 109(4):642-645.
• [4]Holson RR, Pearce B: Principles and pitfalls in the analysis of prenatal treatment effects in multiparous species. Neurotoxicol Teratol 1992, 14(3):221-228.
• [5]Zorrilla EP: Multiparous species present problems (and possibilities) to developmentalists. Dev Psychobiol 1997, 30(2):141-150.
• [6]Wainwright PE: Issues of design and analysis relating to the use of multiparous species in developmental nutritional studies. J Nutr 1998, 128(3):661-663.
• [7]Festing MFW, Altman DG: Guidelines for the design and statistical analysis of experiments using laboratory animals. ILAR J 2002, 43(4):244-258.
• [8]Festing MFW: Principles: the need for better experimental design. Trends Pharmacol Sci 2003, 24(7):341-345.
• [9]Festing MFW: Design and statistical methods in studies using animal models of development. ILAR J 2006, 47:5-14.
• [10]Casella G: Statistical Design. New York: Springer; 2008.
• [11]Maurissen J: Practical considerations on the design, execution and analysis of developmental neurotoxicity studies to be published in Neurotoxicology and Teratology. Neurotoxicol Teratol 2010, 32(2):121-123. [http://dx.doi.org/10.1016/j.ntt.2009.09.002 webcite]
• [12]Lazic SE: The problem of pseudoreplication in neuroscientific studies is it affecting your analysis? BMC Neurosci 2010, 11:5. [http://dx.doi.org/10.1186/1471-2202-11-5 webcite] BioMed Central Full Text
• [13]International Conference on Harmonisation: Detection of toxicity to reproduction for medicinal products and toxicity to male fertility. 1993, S5(R2):1-23. [http://www.ich.org/products/guidelines/safety/safety-single/article/detection-of-toxicity-to-reproduction-for-medicinal-products-toxicity-to-male-fertility.html webcite]
• [14]OECD: Guideline for the testing of chemicals: developmental neurotoxicity study. 2007, 426:1-26. [http://www.oecd-ilibrary.org/test-no-426-developmental-neurotoxicity-study_5l4fg25mnkxs.pdf;jsessionid=12ic71yg7bopl.delta?contentType=/ns/Book&itemId=/content/book/9789264067394-en&containerItemId=/content/serial/ 20745788&accessItemIds=&mimeType=application/pdf webcite]
• [15]Hurlbert SH: Pseudoreplication and the design of ecological field experiments. Ecol Monogr 1984, 54(2):187-211.
• [16]Schnabel J: Neuroscience: Standard model. Nature 2008, 454(7205):682-685. [http://dx.doi.org/10.1038/454682a webcite]
• [17]Scott S, Kranz JE, Cole J, Lincecum JM, Thompson K, Kelly N, Bostrom A, Theodoss J, Al-Nakhala BM, Vieira FG, Ramasubbu J, Heywood JA: Design, power, and interpretation of studies in the standard murine model of ALS. Amyotroph Lateral Scler 2008, 9:4-15. [http://dx.doi.org/10.1080/17482960701856300 webcite]
• [18]Benatar M: Lost in translation: treatment trials in the SOD1 mouse and in human ALS. Neurobiol Dis 2007, 26:1-13. [http://dx.doi.org/10.1016/j.nbd.2006.12.015 webcite]
• [19]Bebarta V, Luyten D, Heard K: Emergency medicine animal research: does use of randomization and blinding affect the results? Acad Emerg Med 2003, 10(6):684-687.
• [20]Crossley NA, Sena E, Goehler J, Horn J, van der Worp B, Bath PMW, Macleod M, Dirnagl U: Empirical evidence of bias in the design of experimental stroke studies: a metaepidemiologic approach. Stroke 2008, 39(3):929-934. [http://dx.doi.org/10.1161/STROKEAHA.107.498725 webcite]
• [21]Kilkenny C, Parsons N, Kadyszewski E, Festing MFW, Cuthill IC, Fry D, Hutton J, Altman DG: Survey of the quality of experimental design, statistical analysis and reporting of research using animals. PLoS One 2009, 4(11):e7824. [http://dx.doi.org/10.1371/journal.pone.0007824 webcite]
• [22]Sena ES, van der Worp HB, Bath PMW, Howells DW, Macleod MR: Publication bias in reports of animal stroke studies leads to major overstatement of efficacy. PLoS Biol 2010, 8(3):e1000344. [http://dx.doi.org/10.1371/journal.pbio.1000344 webcite]
• [23]Vesterinen HM, Sena ES, ffrench Constant C, Williams A, Chandran S, Macleod MR: Improving the translational hit of experimental treatments in multiple sclerosis. Mult Scler 2010, 16(9):1044-1055. [http://dx.doi.org/10.1177/1352458510379612 webcite]
• [24]Hackam DG, Redelmeier DA: Translation of research evidence from animals to humans. JAMA 2006, 296(14):1731-1732. [http://dx.doi.org/10.1001/jama.296.14.1731 webcite]
• [25]Dirnagl U: Bench to bedside: the quest for quality in experimental stroke research. J Cereb Blood Flow Metab 2006, 26(12):1465-1478. [http://dx.doi.org/10.1038/sj.jcbfm.9600298 webcite]
• [26]Philip M, Benatar M, Fisher M, Savitz SI: Methodological quality of animal studies of neuroprotective agents currently in phase II/III, acute ischemic stroke trials. Stroke 2009, 40(2):577-581. [http://dx.doi.org/10.1161/STROKEAHA.108.524330 webcite]
• [27]Bart van der Worp H, Howells DW, Sena ES, Porritt MJ, Rewell S, O’Collins V, Macleod MR: Can animal models of disease reliably inform human studies? PLoS Med 2010, 7(3):e1000245. [http://dx.doi.org/10.1371/journal.pmed.1000245 webcite]
• [28]Shineman DW, Basi GS, Bizon JL, Colton CA, Greenberg BD, Hollister BA, Lincecum J, Leblanc GG, Lee LBH, Luo F, Morgan D, Morse I, Refolo LM, Riddell DR, Scearce-Levie K, Sweeney P, Yrjanheikki J, Fillit HM: Accelerating drug discovery for Alzheimer’s disease: best practices for preclinical animal studies. Alzheimers Res Ther 2011, 3(5):28. http://dx.doi.org/10.1186/alzrt90 webcite BioMed Central Full Text
• [29]Rooke EDM, Vesterinen HM, Sena ES, Egan KJ, Macleod MR: Dopamine agonists in animal models of Parkinson’s disease: a systematic review and meta-analysis. Parkinsonism Relat Disord 2011, 17(5):313-320. [http://dx.doi.org/10.1016/j.parkreldis.2011.02.010 webcite]
• [30]Sarewitz D: Beware the creeping cracks of bias. Nature 2012, 485(7397):149. [http://dx.doi.org/10.1038/485149a webcite]
• [31]Nieuwenhuis S, Forstmann BU, Wagenmakers EJ: Erroneous analyses of interactions in neuroscience: a problem of significance. Nat Neurosci 2011, 14(9):1105-1107. [http://dx.doi.org/10.1038/nn.2886 webcite]
• [32]Cumming G, Fidler F, Vaux DL: Error bars in experimental biology. J Cell Biol 2007, 177:7-11.
• [33]Lazic SE: Relating hippocampal neurogenesis to behavior: the dangers of ignoring confounding variables. Neurobiol Aging 2010, 31(12):2169-2171. [http://dx.doi.org/10.1016/j.neurobiolaging.2010.04.037 webcite]
• [34]Lazic SE: Using causal models to distinguish between neurogenesis-dependent and -independent effects on behaviour. J R Soc Interface 2011. e-pub ahead of print 28 Sep 2011 [http://dx.doi.org/10.1098/rsif.2011.0510 webcite] [http://dx.doi.org/101098/rsif20110510 webcite ]
• [35]Simmons JP, Nelson LD, Simonsohn U: False-positive psychology undisclosed flexibility in data collection and analysis allows presenting anything as significant. Psychol Sci 2011, 22(11):1359-1366. [http://dx.doi.org/10.1177/0956797611417632 webcite]
• [36]Mullard A: Reliability of ’new drug target’ claims called into question. Nat Rev Drug Discov 2011, 10(9):643-644. [http://dx.doi.org/10.1038/nrd3545 webcite]
• [37]Prinz F, Schlange T, Asadullah K: Believe it or not: how much can we rely on published data on potential drug targets? Nat Rev Drug Discov 2011, 10(9):712. [http://dx.doi.org/10.1038/nrd3439-c1 webcite]
• [38]Ledford H: Drug candidates derailed in case of mistaken identity. Nature 2012, 483(7391):519. [http://dx.doi.org/10.1038/483519a webcite]
• [39]Begley CG, Ellis LM: Drug development: Raise standards for preclinical cancer research. Nature 2012, 483(7391):531-533. [http://dx.doi.org/10.1038/483531a webcite]
• [40]Rinaldi T, Silberberg G, Markram H: Hyperconnectivity of local neocortical microcircuitry induced by prenatal exposure to valproic acid. Cereb Cortex 2008, 18(4):763-770. [http://dx.doi.org/10.1093/cercor/bhm117 webcite]
• [41]Mehta MV, Gandal MJ, Siegel SJ: mGluR5-antagonist mediated reversal of elevated stereotyped, repetitive behaviors in the VPA model of autism. PLoS One 2011, 6(10):e26077. [http://dx.doi.org/10.1371/journal.pone.0026077 webcite]
• [42]Zuur AF, Ieno EN, Walker NJ, Saveliev AA, Smith GM: Mixed Effects Models and Extensions in Ecology with R. New York: Springer; 2009.
• [43]Gelman A, Hill J: Data Analysis using Regression and Multilevel/Hierarchical Models. Cambridge, UK: Cambridge University Press; 2007.
• [44]Stodgell CJ, Ingram JL, O’Bara M, Tisdale BK, Nau H, Rodier PM: Induction of the homeotic gene Hoxa1 through valproic acid’s teratogenic mechanism of action. Neurotoxicol Teratol 2006, 28(5):617-624. [http://dx.doi.org/10.1016/j.ntt.2006.08.004 webcite]
• [45]Kuwagata M, Ogawa T, Shioda S, Nagata T: Observation of fetal brain in a rat valproate-induced autism model: a developmental neurotoxicity study. Int J Dev Neurosci 2009, 27(4):399-405. [http://dx.doi.org/10.1016/j.ijdevneu.2009.01.006 webcite]
• [46]Murawski NJ, Brown KL, Stanton ME: Interstimulus interval (ISI) discrimination of the conditioned eyeblink response in a rodent model of autism. Behav Brain Res 2009, 196(2):297-303. [http://dx.doi.org/10.1016/j.bbr.2008.09.020 webcite]
• [47]Flood ZC, Engel DLJ, Simon CC, Negherbon KR, Murphy LJ, Tamavimok W, Anderson GM, Janusonis S: Brain growth trajectories in mouse strains with central and peripheral serotonin differences: relevance to autism models. Neurosci 2012, 210:286-295. [http://dx.doi.org/10.1016/j.neuroscience.2012.03.010 webcite]
• [48]Peers IS, Ceuppens PR, Harbron C: In search of preclinical robustness. Nat Rev Drug Discov 2012, 11(10):733-734. [http://dx.doi.org/10.1038/nrd3849 webcite]
• [49]Landis SC, Amara SG, Asadullah K, Austin CP, Blumenstein R, Bradley EW, Crystal RG, Darnell RB, Ferrante RJ, Fillit H, Finkelstein R, Fisher M, Gendelman HE, Golub RM, Goudreau JL, Gross RA, Gubitz AK, Hesterlee SE, Howells DW, Huguenard J, Kelner K, Koroshetz W, Krainc D, Lazic SE, Levine MS, Macleod MR, McCall JM, Moxley RT, Narasimhan K, Noble LJ, Perrin S, Porter JD, Steward O, Unger E, Utz U, Silberberg SD: A call for transparent reporting to optimize the predictive value of preclinical research. Nature 2012, 490(7419):187-191. [http://dx.doi.org/10.1038/nature11556 webcite]
• [50]Hooijmans CR, Leenaars M, Ritskes-Hoitinga M: A gold standard publication checklist to improve the quality of animal studies, to fully integrate the Three Rs, and to make systematic reviews more feasible. Altern Lab Anim 2010, 38(2):167-182.
• [51]Kilkenny C, Browne WJ, Cuthill IC, Emerson M, Altman DG: Improving bioscience research reporting: the ARRIVE guidelines for reporting animal research. PLoS Biol 2010, 8(6):e1000412. [http://dx.doi.org/10.1371/journal.pbio.1000412 webcite]
• [52]Making methods clearer Nat Neurosci 2013, 16:1. [http://dx.doi.org/10.1038/nn0113-1 webcite]
• [53]Wicherts JM, Borsboom D, Kats J, Molenaar D: The poor availability of psychological research data for reanalysis. Am Psychol 2006, 61(7):726-728. [http://dx.doi.org/10.1037/0003-066X.61.7.726 webcite]
• [54]Bakker M, Wicherts JM: The (mis)reporting of statistical results in psychology journals. Behav Res Methods 2011, 43(3):666-678. [http://dx.doi.org/10.3758/s13428-011-0089-5 webcite]
• [55]Wicherts JM, Bakker M, Molenaar D: Willingness to share research data is related to the strength of the evidence and the quality of reporting of statistical results. PLoS One 2011, 6(11):e26828. [http://dx.doi.org/10.1371/journal.pone.0026828 webcite]