The signed Kolmogorov-Smirnov test: why it should not be used

GigaScience

Guillaume J Filion¹ 

[1] Genome Architecture, Gene Regulation, Stem Cells and Cancer Programme, Centre for Genomic Regulation (CRG), Dr. Aiguader 88, Barcelona, 08003, Spain

关键词: P-hacking;    P-value;    Statistics;    Kolmogorov-smirnov test;   

Others  :  1134021
DOI  :  10.1186/s13742-015-0048-7

 received in 2014-12-02, accepted in 2015-02-03,  发布年份 2015

【 摘 要 】

The two-sample Kolmogorov-Smirnov (KS) test is often used to decide whether two random samples have the same statistical distribution. A popular modification of the KS test is to use a signed version of the KS statistic to infer whether the values of one sample are statistically larger than the values of the other. The underlying hypotheses of the KS test are intrinsically incompatible with this approach and the test can produce false positives supported by extremely low p-values. This potentially makes the signed KS test a tool of p-hacking, which should be discouraged by replacing it with standard tests such as the t-test and by providing confidence intervals instead of p-values.

【 授权许可】

   
2015 Filion; licensee BioMed Central.

【 图 表 】

Figure 1.

Figure 2.

【 参考文献 】

• [1]Lara-Astiaso D, Weiner A, Lorenzo-Vivas E, Zaretsky I, Jaitin DA, David E, et al.: Chromatin state dynamics during blood formation. Science. 2014, 345:943-9.
• [2]Winter EE, Goodstadt L, Ponting CP: Elevated rates of protein secretion, evolution, and disease among tissue-specific genes. Genome Res. 2004, 14:54-61.
• [3]Al-Shahrour F, Minguez P, Tárraga J, Medina I, Alloza E, Montaner D, et al.: FatiGO +: a functional profiling tool for genomic data. Integration of functional annotation, regulatory motifs and interaction data with microarray experiments. Nucleic Acids Res. 2007, 35 Web Server Issue:W91-6.
• [4]Stark KL, Xu B, Bagchi A, Lai WS, Liu H, Hsu R, et al.: Altered brain microRNA biogenesis contributes to phenotypic deficits in a 22q11-deletion mouse model. Nat Genet. 2008, 40:751-60.
• [5]Landan G, Cohen NM, Mukamel Z, Bar A, Molchadsky A, Brosh R, et al.: Epigenetic polymorphism and the stochastic formation of differentially methylated regions in normal and cancerous tissues. Nat Genet. 2012, 44:1207-14.
• [6]Heinz S, Benner C, Spann N, Bertolino E, Lin YC, Laslo P, et al.: Simple combinations of lineage-determining transcription factors prime cis-regulatory elements required for macrophage and B cell identities. Mol Cell. 2010, 38:576-89.