Evaluation of current algorithms for segmentation of scar tissue from late Gadolinium enhancement cardiovascular magnetic resonance of the left atrium: an open-access grand challenge

Journal of Cardiovascular Magnetic Resonance

Kawal Rhode⁷  Rob MacLeod³  Dana Peters⁸  Tobias Schaeffter⁷  Josh Cates³  Daniel Rueckert⁵  Allen Tannenbaum⁴  Reza Razavi⁷  Perry Radau⁶  Heinz-Otto Peitgen¹  Yi Gao²  Wenzhe Shi⁵  Wenjia Bai⁵  YingLi Lu⁶  Anja Hennemuth¹  Samantha Obom⁷  Prince Acheampong⁷  Ebrahim Palkhi⁷  Yosra Al-Beyatti⁷  Ayesha Uddin⁷  Daniel Perry³  Zhong Chen⁷  Mayuragoban Balasubramaniam⁷  R James Housden⁷  Rashed Karim⁷ 

[1] Fraunhofer Institute for Medical Image Computing, Fraunhofer MEVIS, Bremen, Germany;Psychiatry Neuroimaging Lab, Harvard Medical School, Boston, USA;Utah Center for Advanced Imaging Research, University of Utah, Salt Lake City, Utah, USA;School of Electrical and Computer Engineering, Boston University, Boston, USA;Department of Computing, Imperial College London, London, UK;Imaging Research, Sunnybrook Health Sciences Centre, Toronto, Canada;Department of Imaging Sciences & Biomedical Engineering, King’s College London, London, UK;Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, USA

关键词: Algorithm benchmarking;    Segmentation;    Atrial fibrillation;    Cardiovascular magnetic resonance;    Late gadolinium enhancement;   

Others  :  802099
DOI  :  10.1186/1532-429X-15-105

 received in 2013-08-12, accepted in 2013-12-10,  发布年份 2013

【 摘 要 】

Background

Late Gadolinium enhancement (LGE) cardiovascular magnetic resonance (CMR) imaging can be used to visualise regions of fibrosis and scarring in the left atrium (LA) myocardium. This can be important for treatment stratification of patients with atrial fibrillation (AF) and for assessment of treatment after radio frequency catheter ablation (RFCA). In this paper we present a standardised evaluation benchmarking framework for algorithms segmenting fibrosis and scar from LGE CMR images. The algorithms reported are the response to an open challenge that was put to the medical imaging community through an ISBI (IEEE International Symposium on Biomedical Imaging) workshop.

Methods

The image database consisted of 60 multicenter, multivendor LGE CMR image datasets from patients with AF, with 30 images taken before and 30 after RFCA for the treatment of AF. A reference standard for scar and fibrosis was established by merging manual segmentations from three observers. Furthermore, scar was also quantified using 2, 3 and 4 standard deviations (SD) and full-width-at-half-maximum (FWHM) methods. Seven institutions responded to the challenge: Imperial College (IC), Mevis Fraunhofer (MV), Sunnybrook Health Sciences (SY), Harvard/Boston University (HB), Yale School of Medicine (YL), King’s College London (KCL) and Utah CARMA (UTA, UTB). There were 8 different algorithms evaluated in this study.

Results

Some algorithms were able to perform significantly better than SD and FWHM methods in both pre- and post-ablation imaging. Segmentation in pre-ablation images was challenging and good correlation with the reference standard was found in post-ablation images. Overlap scores (out of 100) with the reference standard were as follows: Pre: IC = 37, MV = 22, SY = 17, YL = 48, KCL = 30, UTA = 42, UTB = 45; Post: IC = 76, MV = 85, SY = 73, HB = 76, YL = 84, KCL = 78, UTA = 78, UTB = 72.

Conclusions

The study concludes that currently no algorithm is deemed clearly better than others. There is scope for further algorithmic developments in LA fibrosis and scar quantification from LGE CMR images. Benchmarking of future scar segmentation algorithms is thus important. The proposed benchmarking framework is made available as open-source and new participants can evaluate their algorithms via a web-based interface.

【 授权许可】

   
2013 Karim et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

Figure 7.

Figure 8.

Figure 9.

【 参考文献 】

• [1]Tobon-Gomez C, Craene MD, McLeod K, Tautz L, Shi W, Hennemuth A, Prakosa A, Wang H, Carr-White G, Kapetanakis S, Lutz A, Rasche V, Schaeffter T, Butakoff C, Friman O, Mansi T, Sermesant M, Zhuang X, Ourselin S, Peitgen HO, Pennec X, Razavi R, Rueckert D, Frangi A, Rhode K. Benchmarking framework for myocardial tracking and deformation algorithms: an open access database. Med Image Anal. 2013; 17(6):632-648.
• [2]Kirisli H, Schaap M, Metz C, Dharampall A, Meijboom W, Papadopoulou S, Dedic A, Nieman K, deGraaf M, Meijs M, Cramer M, Broersen A, Cetin S, Eslami A, Flórez-Valencia L, Lor K, Matuszewski B, Melki I, Mohr B, Öksüz I, Shahzad R, Wang C, Kitslaar P, Unal G, Katouzian A, Orkisz M, Chen C, Precioso F, Najman L, Masood S et al.. Standardized evaluation framework for evaluating coronary artery stenosis detection, stenosis quantification and lumen segmentation algorithms in Computed Tomography Angiography. Med Image Anal. 2013; 8(17):856-876.
• [3]Arujuna A, Karim R, Caulfield D, Knowles B, Rhode K, Schaeffter T, Kato B, Rinaldi CA, Cooklin M, Razavi R et al.. Acute pulmonary vein isolation is achieved by a combination of reversible and irreversible atrial injury after catheter ablation: clinical perspective evidence from magnetic Resonance imaging. Circ Arrhythm Electrophysiol. 2012; 5(4):691-700.
• [4]Knowles B, Caulfield D, Cooklin M, Rinaldi C, Gill J, Bostock J, Razavi R, Schaeffter T, Rhode K. 3-D visualization of acute RF ablation lesions using MRI for the simultaneous determination of the patterns of necrosis and edema. Biomed Engineer, IEEE Trans. 2010; 57(6):1467-1475.
• [5]Oakes R, Badger T, Kholmovski E, Akoum N, Burgon N, Fish E, Blauer J, Rao S, DiBella E, Segerson N et al.. Detection and quantification of left atrial structural remodeling with delayed-enhancement magnetic Resonance imaging in patients with atrial fibrillation. Circulation. 2009; 119(13):1758-1767.
• [6]Mahnkopf C, Badger TJ, Burgon NS, Daccarett M, Haslam TS, Badger CT, McGann CJ, Akoum N, Kholmovski E, Macleod RS et al.. Evaluation of the left atrial substrate in patients with lone atrial fibrillation using delayed-enhanced MRI: implications for disease progression and response to catheter ablation. Heart Rhythm. 2010; 7(10):1475-1481.
• [7]Badger TJ, Daccarett M, Akoum NW, Adjei-Poku YA, Burgon NS, Haslam TS, Kalvaitis S, Kuppahally S, Vergara G, McMullen L et al.. Evaluation of left atrial lesions after initial and repeat atrial fibrillation ablation lessons learned from delayed-enhancement MRI in repeat ablation procedures. Circulation: Arrhythmia Electrophysiol. 2010; 3(3):249-259.
• [8]Peters D, Wylie J, Hauser T, Kissinger K, Botnar R, Essebag V, Josephson M, Manning W. Detection of pulmonary vein and left atrial scar after catheter ablation with three-dimensional navigator-gated delayed enhancement MR imaging: initial experience1. Radiol. 2007; 243(3):690.
• [9]Kim R, Fieno D, Parrish T, Harris K, Chen E, Simonetti O, Bundy J, Finn J, Klocke F, Judd R. Relationship of MRI delayed contrast enhancement to irreversible injury, infarct age, and contractile function. Circulation. 1999; 100(19):1992-2002.
• [10]Kolipaka A, Chatzimavroudis GP, White RD, O’Donnell TP, Setser RM. Segmentation of non-viable myocardium in delayed enhancement magnetic resonance images. Int J Cardiovasc Imaging. 2005; 21(2–3):303-311.
• [11]Schmidt A, Azevedo C, Cheng A, Gupta S, Bluemke D, Foo T, Gerstenblith G, Weiss R, Marban E, Tomaselli G et al.. Infarct tissue heterogeneity by magnetic resonance imaging identifies enhanced cardiac arrhythmia susceptibility in patients with left ventricular dysfunction. Circulation. 2007; 115(15):2006-2014.
• [12]Amado L, Gerber B, Gupta S, Rettmann D, Szarf G, Schock R, Nasir K, Kraitchman D, Lima J. Accurate and objective infarct sizing by contrast-enhanced magnetic resonance imaging in a canine myocardial infarction model. J Am College Cardiol. 2004; 44(12):2383-2389.
• [13]Yan A, Shayne A, Brown K, Gupta S, Chan C, Luu T, DiCarli M, Reynolds H, Stevenson W, Kwong R. Characterization of the peri-infarct zone by contrast-enhanced cardiac magnetic resonance imaging is a powerful predictor of post-myocardial infarction mortality. Circulation. 2006; 114:32.
• [14]Positano V, Pingitore A, Giorgetti A, Favilli B, Santarelli MF, Landini L, Marzullo P, Lombardi M. A fast and effective method to assess myocardial necrosis by means of contrast magnetic resonance imaging. J Cardiovasc Magnet Res. 2005; 7(2):487-494.
• [15]Hennemuth A, Seeger A, Friman O, Miller S, Klumpp B, Oeltze S, Peitgen HO. A comprehensive approach to the analysis of contrast enhanced cardiac MR images. Med Imaging, IEEE Trans. 2008; 27(11):1592-1610.
• [16]Detsky J, Paul G, Dick A, Wright G. Reproducible classification of infarct heterogeneity using fuzzy clustering on multicontrast delayed Enhancement Magnetic Resonance Images. Med Imaging, IEEE Trans. 2009; 28(10):1606-1614.
• [17]Tao Q, Milles J, Zeppenfeld K, Lamb HJ, Bax JJ, Reiber JH, van der Geest RJ. Automated segmentation of myocardial scar in late enhancement MRI using combined intensity and spatial information. Magn Reson Med. 2010; 64(2):586-594.
• [18]Lu Y, Yang Y, Connelly KA, Wright GA, Radau PE. Automated quantification of myocardial infarction using graph cuts on contrast delayed enhanced magnetic resonance images. Quant Imaging Med Surg. 2012; 2(2):81.
• [19]Canny J. A computational approach to edge detection. Pattern Anal Mach Intell, IEEE Trans. 1986; 8(6):679-698.
• [20]Goldberg AV, Tarjan RE. A new approach to the maximum-flow problem. J ACM (JACM). 1988; 35(4):921-940.
• [21]Ford LR, Fulkerson DR. Maximal flow through a network. Can J Math. 1956; 8(3):399-404.
• [22]Boykov Y, Veksler O, Zabih R. Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell. 2001; 23(11):1222-1239.
• [23]Boykov Y, Funka-Lea G. Graph cuts and efficient nd image segmentation. Int J Comput Vis. 2006; 70(2):109-131.
• [24]Dunn JC. A fuzzy relative of the ISODATA process and its use in detecting compact well-separated clusters. J Cybernet. 1973; 3(3):32-57.
• [25]Gao Y, Gholami B, MacLeod RS, Blauer J, Haddad WM, Tannenbauma AR. Segmentation of the endocardial wall of the left atrium using local region-based active contours and statistical shape learning. Proc SPIE. 2010; 7623:76234Z1-623418.
• [26]Kass M, Witkin A, Terzopoulos D. Snakes: active contour models. Int J Comp vis. 1988; 1(4):321-331.
• [27]Dempster A, Laird N, Rubin D. Maximum Likelihood from Incomplete data via the EM Algorithm. J R Stat Soc, Ser B (Methodological). 1977; 39:1-38.
• [28]Karim R, Arujuna A, Brazier A, Gill J, Rinaldi CA, O’Neill M, Razavi R, Schaeffter T, Rueckert D, Rhode KS. Automatic segmentation of left atrial scar from delayed-enhancement magnetic resonance imaging. In: International Workshop on Functional Imaging and Modeling of the Heart. Springer; 2011: p.63-70.
• [29]Mitchell TM. Machine Learning, Vol. 45. McGraw Hill: Burr Ridge; 1997.
• [30]Haralick RM, Shanmugam K, Dinstein IH. Textural features for image classification. Syst., Man Cybern, IEEE Trans. 1973; 3(6):610-621.
• [31]Warfield S, Zou K, Wells W. Simultaneous truth and performance level estimation (STAPLE): an algorithm for the validation of image segmentation. Med Imaging, IEEE Trans. 2004; 23(7):903-921.
• [32]Kramer C, Barkhausen J, Flamm S et al.. Standardized CMR protocols, society for cardiovascular magnetic resonance: board of trustees task force on standardized protocols. J Cardiovasc Magn Reson. 2008; 10:35. BioMed Central Full Text
• [33]Flett AS, Hasleton J, Cook C, Hausenloy D, Quarta G, Ariti C, Muthurangu V, Moon JC. Evaluation of techniques for the quantification of myocardial scar of differing etiology using cardiac magnetic resonance. JACC: Cardiovasc Imaging. 2011; 4(2):150-156.
• [34]Lorensen WE, Cline HE. Marching cubes: a high resolution 3D surface construction algorithm. In: ACM Siggraph Computer Graphics, Volume 21. ACM; 1987: p.163-169.
• [35]Hall B, Jeevanantham V, Simon R, Filippone J, Vorobiof G, Daubert J. Variation in left atrial transmural wall thickness at sites commonly targeted for ablation of atrial fibrillation. J Intervent Cardiac Electrophysiol. 2006; 17(2):127-132.