Realistic endoscopic image generation method using virtual-to-real image-domain translation

Healthcare Technology Letters

article

Masahiro Oda¹  Kiyohito Tanaka²  Hirotsugu Takabatake³  Masaki Mori⁴  Hiroshi Natori⁵  Kensaku Mori¹ 

[1] Graduate School of Informatics, Nagoya University;Department of Gastroenterology, Kyoto Second Red Cross Hospital;Department of Respiratory Medicine, Sapporo-Minami-Sanjo Hospital;Department of Respiratory Medicine, Sapporo-Kosei General Hospital;Department of Respiratory Medicine, Keiwakai Nishioka Hospital;Research Center for Medical Bigdata, National Institute of Informatics

关键词: realistic images;    data visualisation;    rendering (computer graphics);    computerised tomography;    endoscopes;    biological organs;    image segmentation;    medical image processing;    virtual reality;    high-quality image-domain translation results;    image cleansing;    image-domain translator;    realistic endoscopic image generation method;    realistic image generation method;    endoscopic simulation systems;    endoscope insertions;    nonrealistic virtual endoscopic images;    unpaired virtual images;    real endoscopic images;    shallow U-Net;    deep U-Net;    endoscopic treatment;    endoscopic diagnosis;   

DOI  :  10.1049/htl.2019.0071

学科分类：肠胃与肝脏病学

来源: Wiley

【 摘 要 】

A realistic image generation method for visualisation in endoscopic simulation systems is proposed in this study. Endoscopic diagnosis and treatment are performed in many hospitals. To reduce complications related to endoscope insertions, endoscopic simulation systems are used for training or rehearsal of endoscope insertions. However, current simulation systems generate non-realistic virtual endoscopic images. To improve the value of the simulation systems, improvement of the reality of their generated images is necessary. The authors propose a realistic image generation method for endoscopic simulation systems. Virtual endoscopic images are generated by using a volume rendering method from a CT volume of a patient. They improve the reality of the virtual endoscopic images using a virtual-to-real image-domain translation technique. The image-domain translator is implemented as a fully convolutional network (FCN). They train the FCN by minimising a cycle consistency loss function. The FCN is trained using unpaired virtual and real endoscopic images. To obtain high-quality image-domain translation results, they perform an image cleansing to the real endoscopic image set. They tested to use the shallow U-Net, U-Net, deep U-Net, and U-Net having residual units as the image-domain translator. The deep U-Net and U-Net having residual units generated quite realistic images.

【 授权许可】

CC BY|CC BY-ND|CC BY-NC|CC BY-NC-ND