Generative multiview inpainting for object removal in large indoor spaces

International Journal of Advanced Robotic Systems

article

Joohyung Kim¹  Janghun Hyeon¹  Nakju Doh² 

[1] School of Electrical Engineering, Korea University;Institute of Convergence Science, Korea University;TeeLabs Inc.

关键词: Multiview inpainting;    object removal;    generative adversarial network;    color consistency;    boundary consistency;   

DOI  :  10.1177/1729881421996544

学科分类：社会科学、人文和艺术（综合）

来源: InTech

【 摘 要 】

As interest in image-based rendering increases, the need for multiview inpainting is emerging. Despite of rapid progresses in single-image inpainting based on deep learning approaches, they have no constraint in obtaining color consistency over multiple inpainted images. We target object removal in large-scale indoor spaces and propose a novel pipeline of multiview inpainting to achieve color consistency and boundary consistency in multiple images. The first step of the pipeline is to create color prior information on masks by coloring point clouds from multiple images and projecting the colored point clouds onto the image planes. Next, a generative inpainting network accepts a masked image, a color prior image, imperfect guideline, and two different masks as inputs and yields the refined guideline and inpainted image as outputs. The color prior and guideline input ensure color and boundary consistencies across multiple images. We validate our pipeline on real indoor data sets quantitatively using consistency distance and similarity distance, metrics we defined for comparing results of multiview inpainting and qualitatively.

【 授权许可】

CC BY