Retinal image synthesis from multiple-landmarks input with generative adversarial networks

BioMedical Engineering OnLine

Yanye Lu¹  Qiushi Ren²  Zekuan Yu²  Qing Xiang³  Jiahao Meng³  Caixia Kou³ 

[1] 0000 0001 2107 3311, grid.5330.5, Pattern Recognition Lab, Department of Computer Science, Friedrich-Alexander-University Erlangen-Nuremberg, 91058, Erlangen, Germany;0000 0001 2256 9319, grid.11135.37, Department of Biomedical Engineering, College of Engineering, Peking University, 100871, Beijing, China;grid.31880.32, Beijing University of Posts and Telecommunications, 100876, Beijing, China;

关键词: Retinal image synthesis;    Generative adversarial networks;    Multiple landmarks;   

DOI  :  10.1186/s12938-019-0682-x

来源: publisher

【 摘 要 】

BackgroundMedical datasets, especially medical images, are often imbalanced due to the different incidences of various diseases. To address this problem, many methods have been proposed to synthesize medical images using generative adversarial networks (GANs) to enlarge training datasets for facilitating medical image analysis. For instance, conventional methods such as image-to-image translation techniques are used to synthesize fundus images with their respective vessel trees in the field of fundus image.MethodsIn order to improve the image quality and details of the synthetic images, three key aspects of the pipeline are mainly elaborated: the input mask, architecture of GANs, and the resolution of paired images. We propose a new preprocessing pipeline named multiple-channels-multiple-landmarks (MCML), aiming to synthesize color fundus images from a combination of vessel tree, optic disc, and optic cup images. We compared both single vessel mask input and MCML mask input on two public fundus image datasets (DRIVE and DRISHTI-GS) with different kinds of Pix2pix and Cycle-GAN architectures. A new Pix2pix structure with ResU-net generator is also designed, which has been compared with the other models.Results and conclusionAs shown in the results, the proposed MCML method outperforms the single vessel-based methods for each architecture of GANs. Furthermore, we find that our Pix2pix model with ResU-net generator achieves superior PSNR and SSIM performance than the other GANs. High-resolution paired images are also beneficial for improving the performance of each GAN in this work. Finally, a Pix2pix network with ResU-net generator using MCML and high-resolution paired images are able to generate good and realistic fundus images in this work, indicating that our MCML method has great potential in the field of glaucoma computer-aided diagnosis based on fundus image.

【 授权许可】

CC BY