Local Style Preservation in Improved GAN-Driven Synthetic Image Generation for Endoscopic Tool Segmentation

Sensors

卷:21

Wenfan Jiang¹  Yun-Hsuan Su¹  Haonan Peng²  Blake Hannaford²  Digesh Chitrakar³  Kevin Huang³ 

[1] Department of Computer Science, Mount Holyoke College, 50 College Street, South Hadley, MA 01075, USA;

[2] Department of Electrical and Computer Engineering, University of Washington, 185 Stevens Way, Paul Allen Center, Seattle, WA 98105, USA;

[3] Department of Engineering, Trinity College, 300 Summit St., Hartford, CT 06106, USA;

关键词: robot-assisted minimally invasive surgery;    surgical tool segmentation;    generative adversarial networks;    UNet;    medical imaging;   

DOI  :  10.3390/s21155163

来源: DOAJ

【 摘 要 】

Accurate semantic image segmentation from medical imaging can enable intelligent vision-based assistance in robot-assisted minimally invasive surgery. The human body and surgical procedures are highly dynamic. While machine-vision presents a promising approach, sufficiently large training image sets for robust performance are either costly or unavailable. This work examines three novel generative adversarial network (GAN) methods of providing usable synthetic tool images using only surgical background images and a few real tool images. The best of these three novel approaches generates realistic tool textures while preserving local background content by incorporating both a style preservation and a content loss component into the proposed multi-level loss function. The approach is quantitatively evaluated, and results suggest that the synthetically generated training tool images enhance UNet tool segmentation performance. More specifically, with a random set of 100 cadaver and live endoscopic images from the University of Washington Sinus Dataset, the UNet trained with synthetically generated images using the presented method resulted in 35.7% and 30.6% improvement over using purely real images in mean Dice coefficient and Intersection over Union scores, respectively. This study is promising towards the use of more widely available and routine screening endoscopy to preoperatively generate synthetic training tool images for intraoperative UNet tool segmentation.

【 授权许可】

Unknown