Manipulating the Alpha Level Cannot Cure Significance Testing

Frontiers in Psychology

Isabel Suarez¹  Michael T. Bradley²  Marion Tegethoff³  Marian Grendar⁴  Juan J. Rahona⁵  Martin Lachmair⁵  Ladislas Nalborczyk⁸  James W. Grice⁹  Mauricio Tejo^1,10  William M. Briggs^1,11  Hung T. Nguyen^1,12  Rubén D. Ledesma^1,14  Andrés Gutiérrez^1,15  Héctor A. Cepeda-Freyre^1,16  Sergio E. Chaigneau^1,17  Raydonal Ospina^1,18  Tania B. Huedo-Medina^1,19  Rens van de Schoot^2,20  Jose D. Perezgonzalez^2,22  Klaus Jaffe^2,23  Ivan I. Vankov^2,24  Daniel R. Ciocca^2,25  Martin E. Guerrero-Gimenez^2,25  Fernando Marmolejo-Ramos^2,25  Roberto Limongi^2,26  Santiago Velasco-Forero^2,27  Marco T. Liuzza^2,28  Roland Pfister^2,29  Tonghui Wang^3,30  Juan C. Correa^3,31  Yuki Yamada^3,32  Rosaria Lombardo^3,33  Armina Janyan^3,34  David A. Rodríguez-Medina^3,35  Denis Cousineau^3,36  Xavier Romão^3,37  Ali Karimnezhad^3,38  Michiel R. de Boer^3,39  Felipe C. M. Zoppino^4,41  Subhra S. Dhar^4,42  Fränzi Korner-Nievergelt^4,45  Gunther Meinlschmidt^4,47  Susana Ruiz-Fernández^4,48  Koji Kosugi^4,49  Corson N. Areshenkoff^5,50  Yusuf K. Bilgiç^5,51  David Trafimow^5,52  Igor Dolgov^5,52  Michael J. Marks^5,52  Carlos J. Barrera-Causil^5,53  Juana Gómez-Benito^5,54  Roser Bono^5,54  Eric J. Beh^5,56  Valentin Amrhein^5,57 

[1] Research Network, University of Tübingen, Tübingen, Germany;0Department of Psychology, Faculty of Arts, University of New Brunswick, Saint John, NB, Canada;0Department of Psychology, Universidad del Norte, Barranquilla, Colombia;0Institute of Measurement Science, Slovak Academy of Sciences, Bratislava, Slovakia;0Multimodal Interaction Lab, Leibniz-Institut für Wissensmedien, Tübingen, Germany;0Université Grenoble Alpes, Centre National de la Recherche Scientifique, LPNC, Grenoble, France;1Consejo Nacional de Investigaciones Científicas y Técnicas, Mar del Plata, Argentina;1Department of Experimental Clinical and Health Psychology, Ghent University, Ghent, Belgium;1Department of Psychology, Oklahoma State University, Stillwater, OK, United States;1Division of Clinical Psychology and Psychiatry, Department of Psychology, University of Basel, Basel, Switzerland;1Independent Researcher, New York, NY, United States;2Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States;2Facultad de Ciencias Naturales y Exactas, Universidad de Playa Ancha, Valparaíso, Chile;2Facultad de Psicología, Universidad Nacional de Mar del Plata, Mar del Plata, Argentina;2Faculty of Statistics, Saint Thomas University, Bogotá, Colombia;2School of Psychology, Benemérita Universidad Autónoma de Puebla, Puebla, Mexico;3Center for Social and Cognitive Neuroscience, School of Psychology, Universidad Adolfo Ibáñez, Santiago, Chile;3Computational Statistics Laboratory (CAST), Department of Statistics, Universidade Federal de Pernambuco, Recife, Brazil;3Department of Allied Health Sciences, College of Health, Agriculture, and Natural Resources, University of Connecticut, Storrs, CT, United States;3Department of Methods and Statistics, Faculty of Social and Behavioural Sciences, Utrecht University, Utrecht, Netherlands;3Pontificia Universidad Católica de Valparaíso, Valparaíso, Chile;4Business School, Massey University, Albany, New Zealand;4Departamento de Biología de Organismos, Universidad Simón Bolívar, Caracas, Venezuela;4North-West University, Optentia Research Focus Area, Vanderbijlpark, South Africa;4Oncology Laboratory, Instituto de Medicina y Biologia Experimental de Cuyo, CCT CONICET Mendoza, Mendoza, Argentina;4Vicerrectoría de Investigación y Desarrollo, Universidad Tecnológica de Chile INACAP, Santiago, Chile;5Department of Cognitive Science and Psychology, New Bulgarian University, Sofia, Bulgaria;5Department of Medical and Surgical Sciences, “Magna Graecia” University of Catanzaro, Catanzaro, Italy;5Department of Psychology III, University of Würzburg, Würzburg, Germany;5MINES Paristech, PSL Research University, Centre for Mathematical Morphology, Paris, France;5School of Statistics, Faculty of Sciences, National University of Colombia, Medellín, Colombia;6Department of Mathematical Sciences, New Mexico State University, Las Cruces, NM, United States;6Economics Department, University of Campania “Luigi Vanvitelli”, Capua, Italy;6National Research Tomsk State University, Tomsk, Russia;6School of Psychology, National Autonomous University of Mexico, Mexico City, Mexico;6School of Psychology, University of Ottawa, Ottawa, ON, Canada;7CONSTRUCT-LESE, Faculty of Engineering, University of Porto, Porto, Portugal;7Department of Biochemistry, Microbiology, and Immunology, University of Ottawa, Ottawa, ON, Canada;7Department of Health Sciences, Vrije Universiteit Amsterdam and Amsterdam Public Health Research Institute, Amsterdam, Netherlands;7Department of Psychosomatic Medicine, University Hospital Basel and University of Basel, Basel, Switzerland;7Faculty of Arts and Science, Kyushu University, Fukuoka, Japan;8Department of Mathematics and Statistics, Indian Institute of Technology, Kanpur, India;8Division of Clinical Psychology and Cognitive Behavioral Therapy, International Psychoanalytic University, Berlin, Germany;8FOM Hochschule für Oekonomie und Management, Essen, Germany;8Oikostat GmbH, Ettiswil, Switzerland;9Biomedical Center Martin, Jessenius Faculty of Medicine, Comenius University, Martin, Slovakia;9Division of Clinical Psychology and Epidemiology, Department of Psychology, University of Basel, Basel, Switzerland;;9LEAD Graduate School &9School of Human Sciences, Senshu University, Kawasaki, Japan;Centre for Neuroscience Studies, Queens University, Kingston, ON, Canada;Department of Mathematics, State University of New York at Geneseo, Geneseo, NY, United States;Department of Psychology, New Mexico State University, Las Cruces, NM, United States;Faculty of Applied and Exact Sciences, Metropolitan Technological Institute, Medellín, Colombia;Institut de Neurociències, University of Barcelona, Barcelona, Spain;Quantitative Psychology Unit, Faculty of Psychology, University of Barcelona, Barcelona, Spain;School of Mathematical and Physical Sciences, University of Newcastle, Callaghan, NSW, Australia;Swiss Ornithological Institute, Sempach, Switzerland;Zoological Institute, University of Basel, Basel, Switzerland;

关键词: statistical significance;    null hypothesis testing;    p-value;    significance testing;    decision making;   

DOI  :  10.3389/fpsyg.2018.00699

来源: DOAJ

【 摘 要 】

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new discoveries and the progress of science. Given that blanket and variable alpha levels both are problematic, it is sensible to dispense with significance testing altogether. There are alternatives that address study design and sample size much more directly than significance testing does; but none of the statistical tools should be taken as the new magic method giving clear-cut mechanical answers. Inference should not be based on single studies at all, but on cumulative evidence from multiple independent studies. When evaluating the strength of the evidence, we should consider, for example, auxiliary assumptions, the strength of the experimental design, and implications for applications. To boil all this down to a binary decision based on a p-value threshold of 0.05, 0.01, 0.005, or anything else, is not acceptable.

【 授权许可】

Unknown