Planned Hypothesis Tests Are Not Necessarily Exempt From Multiplicity Adjustment

Journal of research practice

Andrew V. Frane¹ 

[1] University of California Los Angeles

关键词: hypothesis testing;    null hypothesis;    statistical inference;    statistical methods;    hypothesis testing;    Type I error;   

学科分类：社会科学、人文和艺术（综合）

来源: Journal of research practice

【 摘 要 】

Scientific research often involves testing more than one hypothesis at a time, which can inflate the probability that a Type I error (false discovery) will occur. To prevent this Type I error inflation, adjustments can be made to the testing procedure that compensate for the number of tests. Yet many researchers believe that such adjustments are inherently unnecessary if the tests were “planned” (i.e., if the hypotheses were specified before the study began). This longstanding misconception continues to be perpetuated in textbooks and continues to be cited in journal articles to justify disregard for Type I error inflation. I critically evaluate this myth and examine its rationales and variations. To emphasize the myth’s prevalence and relevance in current research practice, I provide examples from popular textbooks and from recent literature. I also make recommendations for improving research practice and pedagogy regarding this problem and regarding multiple testing in general.

【 授权许可】

CC BY-NC-ND