A novel measure and significance testing in data analysis of cell image segmentation

BMC Bioinformatics

Research Article

Anne L. Plant¹  Jin Chu Wu¹  John T. Elliott¹  Raghu N. Kacker¹  Michael Halter¹ 

[1] National Institute of Standards and Technology, 20899, Gaithersburg, MD, USA;

关键词: Cell image segmentation;    Cell assays;    Performance measure;    Misclassification error rate;    Total error rate;    Standard error;    Confidence interval;    Correlation coefficient;    Significance testing;    Bootstrap method;   

DOI  :  10.1186/s12859-017-1527-x

 received in 2016-08-11, accepted in 2017-02-06,  发布年份 2017

来源: Springer

【 摘 要 】

BackgroundCell image segmentation (CIS) is an essential part of quantitative imaging of biological cells. Designing a performance measure and conducting significance testing are critical for evaluating and comparing the CIS algorithms for image-based cell assays in cytometry. Many measures and methods have been proposed and implemented to evaluate segmentation methods. However, computing the standard errors (SE) of the measures and their correlation coefficient is not described, and thus the statistical significance of performance differences between CIS algorithms cannot be assessed.ResultsWe propose the total error rate (TER), a novel performance measure for segmenting all cells in the supervised evaluation. The TER statistically aggregates all misclassification error rates (MER) by taking cell sizes as weights. The MERs are for segmenting each single cell in the population. The TER is fully supported by the pairwise comparisons of MERs using 106 manually segmented ground-truth cells with different sizes and seven CIS algorithms taken from ImageJ. Further, the SE and 95% confidence interval (CI) of TER are computed based on the SE of MER that is calculated using the bootstrap method. An algorithm for computing the correlation coefficient of TERs between two CIS algorithms is also provided. Hence, the 95% CI error bars can be used to classify CIS algorithms. The SEs of TERs and their correlation coefficient can be employed to conduct the hypothesis testing, while the CIs overlap, to determine the statistical significance of the performance differences between CIS algorithms.ConclusionsA novel measure TER of CIS is proposed. The TER’s SEs and correlation coefficient are computed. Thereafter, CIS algorithms can be evaluated and compared statistically by conducting the significance testing.

【 授权许可】

CC BY   
© The Author(s). 2017

【 图 表 】

12888_2016_877_Article_IEq13.gif

12888_2016_877_Article_IEq12.gif

12888_2016_877_Article_IEq10.gif

12864_2015_2300_Article_IEq24.gif

12864_2017_3487_Article_IEq15.gif

12864_2017_4166_Article_IEq3.gif

12864_2016_3105_Article_IEq15.gif

12864_2017_3733_Article_IEq13.gif

12864_2017_3669_Article_IEq5.gif

12864_2016_2789_Article_IEq16.gif

12864_2017_3669_Article_IEq4.gif

12906_2017_1593_Article_IEq25.gif

12864_2016_3105_Article_IEq10.gif

12906_2015_Article_682_TeX2GIF_IEq2.gif

12864_2016_3353_Article_IEq8.gif