Testing Significance Testing

Collabra: Psychology

Joachim I. Krueger¹  Patrick R. Heck² 

[1] Department of Cognitive, Linguistic and Psychological Sciences, Brown University, Providence, RI;Geisinger Health System, Danville, PA

关键词: statistical significance testing;    null hypotheses;    Bayes’;    Theorem;    NHST;    p values;   

DOI  :  10.1525/collabra.108

学科分类：社会科学、人文和艺术（综合）

来源: University of California Press

【 摘 要 】

The practice of Significance Testing (ST) remains widespread in psychological science despite continual criticism of its flaws and abuses. Using simulation experiments, we address four concerns about ST and for two of these we compare ST’s performance with prominent alternatives. We find the following: First, the p values delivered by ST predict the posterior probability of the tested hypothesis well under many research conditions. Second, low p values support inductive inferences because they are most likely to occur when the tested hypothesis is false. Third, p values track likelihood ratios without raising the uncertainties of relative inference. Fourth, p values predict the replicability of research findings better than confidence intervals do. Given these results, we conclude that p values may be used judiciously as a heuristic tool for inductive inference. Yet, p values cannot bear the full burden of inference. We encourage researchers to be flexible in their selection and use of statistical methods.

.

【 授权许可】

CC BY