Reporting and interpreting non-significant results in animal cognition research

PeerJ

article

Benjamin G. Farrar¹  Alizée Vernouillet³  Elias Garcia-Pelegrin¹  Edward W. Legg⁵  Katharina F. Brecht⁸  Poppy J. Lambert⁹  Mahmoud Elsherif^1,10  Shannon Francis^1,12  Laurie O’Neill^1,12  Nicola S. Clayton¹  Ljerka Ostojić⁵ 

[1] Department of Psychology, University of Cambridge;Institute for Globally Distributed Open Research and Education;Department of Experimental Psychology, Universiteit Gent;Department of Psychology, National University of Singapore;Department of Psychology, Faculty of Humanities and Social Sciences, University of Rijeka;Division of Cognitive Sciences, University of Rijeka;Centre for Mind and Behaviour, University of Rijeka;Institute for Neurobiology, University of Tuebingen;Messerli Research Insititute, University of Vienna;Department of Psychology, University of Birmingham;University of Leicester;Comparative Cognition Research Group, Max Planck Institute for Ornithology

关键词: Animal behavior;    Animal cognition;    Null hypothesis significance testing;    Non-significant results;    Statistical inferences;    Negative results;   

DOI  :  10.7717/peerj.14963

学科分类：社会科学、人文和艺术（综合）

来源: Inra

【 摘 要 】

How statistically non-significant results are reported and interpreted following null hypothesis significance testing is often criticized. This issue is important for animal cognition research because studies in the field are often underpowered to detect theoretically meaningful effect sizes, i.e., often produce non-significant p-values even when the null hypothesis is incorrect. Thus, we manually extracted and classified how researchers report and interpret non-significant p-values and examined the p-value distribution of these non-significant results across published articles in animal cognition and related fields. We found a large amount of heterogeneity in how researchers report statistically non-significant p-values in the result sections of articles, and how they interpret them in the titles and abstracts. Reporting of the non-significant results as “No Effect” was common in the titles (84%), abstracts (64%), and results sections (41%) of papers, whereas reporting of the results as “Non-Significant” was less common in the titles (0%) and abstracts (26%), but was present in the results (52%). Discussions of effect sizes were rare (<5% of articles). A p-value distribution analysis was consistent with research being performed with low power of statistical tests to detect effect sizes of interest. These findings suggest that researchers in animal cognition should pay close attention to the evidence used to support claims of absence of effects in the literature, and—in their own work—report statistically non-significant results clearly and formally correct, as well as use more formal methods of assessing evidence against theoretical predictions.

【 授权许可】

CC BY