Multilevel analysis quantifies variation in the experimental effect while optimizing power and preventing false positives

BMC Neuroscience

Research Article

Conor V. Dolan¹  Sophie van der Sluis²  Matthijs Verhage³  Emmeke Aarts⁴ 

[1] Department of Biological Psychology, VU University Amsterdam, Van der Boechorststraat 1, 1081 BT, Amsterdam, The Netherlands;Department of Clinical Genetics, Section Complex Trait Genetics, VU Medical Center, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Clinical Genetics, Section Functional Genomics, VU Medical Center Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Functional Genomics, Center for Neurogenomics and Cognitive Research, VU University Amsterdam, De Boelelaan 1085, 1081 HV, Amsterdam, The Netherlands;Department of Molecular Computational Biology, Max Planck Institute of Molecular Genetics, Ihnestraße 63-73, 14195, Berlin, Germany;

关键词: Multilevel analysis;    False positive rate;    Pseudo-replication;    Statistical power;    Hierarchical data;    Clustered data;    Optimal research design;    Experimental effect;    Neuroscience;   

DOI  :  10.1186/s12868-015-0228-5

 received in 2015-02-20, accepted in 2015-12-01,  发布年份 2015

来源: Springer

【 摘 要 】

BackgroundIn neuroscience, experimental designs in which multiple measurements are collected in the same research object or treatment facility are common. Such designs result in clustered or nested data. When clusters include measurements from different experimental conditions, both the mean of the dependent variable and the effect of the experimental manipulation may vary over clusters. In practice, this type of cluster-related variation is often overlooked. Not accommodating cluster-related variation can result in inferential errors concerning the overall experimental effect.ResultsThe exact effect of ignoring the clustered nature of the data depends on the effect of clustering. Using simulation studies we show that cluster-related variation in the experimental effect, if ignored, results in a false positive rate (i.e., Type I error rate) that is appreciably higher (up to ~20–~50 %) than the chosen α\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\alpha$$\end{document}-level (e.g., α\documentclass[12pt]{minimal}\usepackage{amsmath}\usepackage{wasysym}\usepackage{amsfonts}\usepackage{amssymb}\usepackage{amsbsy}\usepackage{mathrsfs}\usepackage{upgreek}\setlength{\oddsidemargin}{-69pt}\begin{document}$$\alpha$$\end{document} = 0.05). If the effect of clustering is limited to the intercept, the failure to accommodate clustering can result in a loss of statistical power to detect the overall experimental effect. This effect is most pronounced when both the magnitude of the experimental effect and the sample size are small (e.g., ~25 % less power given an experimental effect with effect size d of 0.20, and a sample size of 10 clusters and 5 observations per experimental condition per cluster).ConclusionsWhen data is collected from a research design in which observations from the same cluster are obtained in different experimental conditions, multilevel analysis should be used to analyze the data. The use of multilevel analysis not only ensures correct statistical interpretation of the overall experimental effect, but also provides a valuable test of the generalizability of the experimental effect over (intrinsically) varying settings, and a means to reveal the cause of cluster-related variation in experimental effect.

【 授权许可】

CC BY   
© Aarts et al. 2015

【 图 表 】

Fig. 2

Fig. 4

Fig. 1

Fig. 3

Fig. 3

12947_2017_100_Article_IEq1.gif

Fig. 1

Fig. 1

Fig. 3

12951_2015_155_Article_IEq70.gif

Fig. 10

Fig. 2

Fig. 7

Fig. 5

Fig. 1

Fig. 19

12951_2017_255_Article_IEq45.gif

Fig. 3

Fig. 6

Fig. 4

12951_2015_155_Article_IEq69.gif

Fig. 18

12888_2023_5206_Article_IEq3.gif

Fig. 17

12888_2023_5206_Article_IEq1.gif

Fig. 16

Fig. 1

Fig. 15

Fig. 3

Fig. 14

Fig. 5

Fig. 2

Fig. 13

Fig. 1

Fig. 1

Fig. 1

Fig. 12

Fig. 6

Fig. 3

Fig. 11

Fig. 2

Fig. 9

Fig. 1

12936_2017_1963_Article_IEq41.gif

Fig. 8

Fig. 8

Fig. 5

Fig. 3

Fig. 1

Fig. 2

Fig. 1

12936_2015_836_Article_IEq14.gif

12951_2017_255_Article_IEq41.gif

12936_2015_836_Article_IEq13.gif

Fig. 2

12936_2017_2014_Article_IEq29.gif

Fig. 2

Fig. 4

12936_2017_2045_Article_IEq18.gif

Fig. 2

Fig. 2