Multilevel analysis quantifies variation in the experimental effect while optimizing power and preventing false positives

BMC Neuroscience

Research Article

Conor V. Dolan¹  Sophie van der Sluis²  Matthijs Verhage³  Emmeke Aarts⁴ 

[1] Department of Biological Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT, Amsterdam, The Netherlands;Department of Clinical Genetics, Section Complex Trait Genetics, VU Medical Center, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Clinical Genetics, Section Functional Genomics, VU Medical Center Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Molecular Computational Biology, Max Planck Institute of Molecular Genetics, Ihnestraße 63-73, 14195, Berlin, Germany;

关键词: Multilevel analysis;    False positive rate;    Pseudo-replication;    Statistical power;    Hierarchical data;    Clustered data;    Optimal research design;    Experimental effect;    Neuroscience;   

DOI  :  10.1186/s12868-015-0228-5

 received in 2015-02-20, accepted in 2015-12-01,  发布年份 2015

来源: Springer

【 摘 要 】

BackgroundIn neuroscience, experimental designs in which multiple measurements are collected in the same research object or treatment facility are common. Such designs result in clustered or nested data. When clusters include measurements from different experimental conditions, both the mean of the dependent variable and the effect of the experimental manipulation may vary over clusters. In practice, this type of cluster-related variation is often overlooked. Not accommodating cluster-related variation can result in inferential errors concerning the overall experimental effect.ResultsThe exact effect of ignoring the clustered nature of the data depends on the effect of clustering. Using simulation studies we show that cluster-related variation in the experimental effect, if ignored, results in a false positive rate (i.e., Type I error rate) that is appreciably higher (up to ~20–~50 %) than the chosen α\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\alpha$$\end{document}-level (e.g., α\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\alpha$$\end{document} = 0.05). If the effect of clustering is limited to the intercept, the failure to accommodate clustering can result in a loss of statistical power to detect the overall experimental effect. This effect is most pronounced when both the magnitude of the experimental effect and the sample size are small (e.g., ~25 % less power given an experimental effect with effect size d of 0.20, and a sample size of 10 clusters and 5 observations per experimental condition per cluster).ConclusionsWhen data is collected from a research design in which observations from the same cluster are obtained in different experimental conditions, multilevel analysis should be used to analyze the data. The use of multilevel analysis not only ensures correct statistical interpretation of the overall experimental effect, but also provides a valuable test of the generalizability of the experimental effect over (intrinsically) varying settings, and a means to reveal the cause of cluster-related variation in experimental effect.

【 授权许可】

CC BY   
© Aarts et al. 2015

【 图 表 】

12864_2016_3477_Article_IEq15.gif

12864_2015_2300_Article_IEq19.gif

12864_2017_3733_Article_IEq12.gif

12864_2015_2295_Article_IEq3.gif

12864_2016_3440_Article_IEq53.gif

12902_2017_191_Article_IEq3.gif

12864_2017_3669_Article_IEq4.gif

12888_2016_877_Article_IEq2.gif

12906_2017_1593_Article_IEq25.gif

12888_2016_877_Article_IEq1.gif

12864_2015_2055_Article_IEq9.gif

12864_2017_4132_Article_IEq1.gif

12864_2015_2055_Article_IEq8.gif

12864_2017_4186_Article_IEq26.gif

12888_2016_811_Article_IEq1.gif

12864_2017_4004_Article_IEq7.gif

12864_2017_4030_Article_IEq20.gif

12864_2017_4004_Article_IEq6.gif

12711_2017_362_Article_IEq93.gif

12864_2015_2170_Article_IEq3.gif

12880_2015_Article_74_TeX2GIF_IEq3.gif

12864_2017_3669_Article_IEq3.gif

12902_2017_191_Article_IEq1.gif

12864_2017_3669_Article_IEq2.gif

12864_2016_3105_Article_IEq10.gif

12864_2017_3669_Article_IEq1.gif

12864_2016_3353_Article_IEq8.gif

12864_2017_4186_Article_IEq15.gif

12864_2017_4186_Article_IEq1.gif

12864_2017_4179_Article_IEq9.gif

12864_2017_4179_Article_IEq8.gif

12864_2017_3836_Article_IEq2.gif

12864_2017_3731_Article_IEq3.gif

12864_2017_4030_Article_IEq13.gif

12864_2017_3527_Article_IEq9.gif

12880_2015_Article_54_TeX2GIF_IEq16.gif

12864_2016_2682_Article_IEq50.gif

12864_2017_3527_Article_IEq6.gif

12864_2017_4274_Article_IEq11.gif

12864_2016_2682_Article_IEq48.gif

12864_2016_3098_Article_IEq57.gif

12888_2017_1504_Article_IEq2.gif

12864_2016_3265_Article_IEq1.gif

12864_2016_3477_Article_IEq5.gif

12864_2017_3670_Article_IEq2.gif

12864_2017_3500_Article_IEq24.gif

12864_2015_2055_Article_IEq84.gif

12864_2017_4191_Article_IEq1.gif

12864_2017_3487_Article_IEq6.gif

12864_2017_3500_Article_IEq22.gif

12864_2017_3496_Article_IEq2.gif

12864_2017_3938_Article_IEq1.gif

12864_2017_3500_Article_IEq20.gif

12864_2017_3500_Article_IEq19.gif

12864_2017_4020_Article_IEq49.gif

12864_2017_4020_Article_IEq48.gif

12864_2017_4025_Article_IEq10.gif

12864_2017_3655_Article_IEq24.gif

12864_2016_2821_Article_IEq7.gif

12864_2016_3440_Article_IEq27.gif

12864_2016_2871_Article_IEq4.gif

12864_2017_4030_Article_IEq1.gif

12864_2016_2443_Article_IEq14.gif

12864_2017_4025_Article_IEq2.gif

12864_2016_2443_Article_IEq13.gif

12864_2016_2443_Article_IEq12.gif

12864_2016_3426_Article_IEq2.gif

12864_2015_1994_Article_IEq17.gif

12864_2017_4130_Article_IEq5.gif

12864_2015_1994_Article_IEq16.gif

12864_2017_4309_Article_IEq14.gif

12864_2015_2163_Article_IEq3.gif

12864_2015_2296_Article_IEq95.gif

12864_2017_3655_Article_IEq23.gif

12864_2016_3210_Article_IEq1.gif