Improved assessment of multiple sclerosis lesion segmentation agreement via detection and outline error estimates

BMC Medical Imaging

Technical Advance

Guy Poloni¹  Sara Hussein¹  Niels Bergsland¹  Deepa Ramasamy¹  Michael G Dwyer¹  David S Wack²  Robert Zivadinov³  Laura Ranza⁴  Carol Di Perri⁴ 

[1] Buffalo Neuroimaging Analysis Center, Dept. of Neurology, University at Buffalo, State University of New York at Buffalo, Buffalo, NY, USA;Buffalo Neuroimaging Analysis Center, Dept. of Neurology, University at Buffalo, State University of New York at Buffalo, Buffalo, NY, USA;Department of Nuclear Medicine, University at Buffalo, State University of New York at Buffalo, Buffalo, NY, USA;Buffalo Neuroimaging Analysis Center, Jacobs Neurological Institute, State University of NY at Buffalo, 100 High St., 14203, Buffalo, NY, USA;Buffalo Neuroimaging Analysis Center, Dept. of Neurology, University at Buffalo, State University of New York at Buffalo, Buffalo, NY, USA;The Jacobs Neurological Institute, Dept. of Neurology, University at Buffalo, State University of New York at Buffalo, Buffalo, NY, USA;Department of Neuroradiology, IRCCS, C. Mondino, University of Pavia, Pavia, Italy;

关键词: Multiple sclerosis;    Detection and outline error estimates;    Rater agreement;    Operator agreement;    Metric;    Jaccard Index;    Similarity index;    Measure;    Index;    Kappa;    Lesion;    MRI;    ROI;   

DOI  :  10.1186/1471-2342-12-17

 received in 2011-11-08, accepted in 2012-07-19,  发布年份 2012

来源: Springer

【 摘 要 】

BackgroundPresented is the method “Detection and Outline Error Estimates” (DOEE) for assessing rater agreement in the delineation of multiple sclerosis (MS) lesions. The DOEE method divides operator or rater assessment into two parts: 1) Detection Error (DE) -- rater agreement in detecting the same regions to mark, and 2) Outline Error (OE) -- agreement of the raters in outlining of the same lesion.MethodsDE, OE and Similarity Index (SI) values were calculated for two raters tested on a set of 17 fluid-attenuated inversion-recovery (FLAIR) images of patients with MS. DE, OE, and SI values were tested for dependence with mean total area (MTA) of the raters' Region of Interests (ROIs).ResultsWhen correlated with MTA, neither DE (ρ = .056, p=.83) nor the ratio of OE to MTA (ρ = .23, p=.37), referred to as Outline Error Rate (OER), exhibited significant correlation. In contrast, SI is found to be strongly correlated with MTA (ρ = .75, p < .001). Furthermore, DE and OER values can be used to model the variation in SI with MTA.ConclusionsThe DE and OER indices are proposed as a better method than SI for comparing rater agreement of ROIs, which also provide specific information for raters to improve their agreement.

【 授权许可】

CC BY   
© Wack et al.; licensee BioMed Central Ltd. 2012