Benefiting from multitask learning to improve single image super-resolution

NEUROCOMPUTING

卷:398

Article

Rad, Mohammad Saeed¹  Bozorgtabar, Behzad¹  Musat, Claudiu²  Marti, Urs-Viktor²  Basler, Max²  Ekenel, Hazim Kemal^1,3  Thiran, Jean-Philippe¹ 

[1] Ecole Polytech Fed Lausanne EPFL, Signal Proc Lab 5, Lausanne, Switzerland

[2] Swisscom AG, AI Lab, Lausanne, Switzerland

[3] Istanbul Tech Univ, Istanbul, Turkey

关键词: Single image super-resolution;    Multitask learning;    Recovering realistic textures;    Semantic segmentation;    Generative adversarial network;   

DOI  :  10.1016/j.neucom.2019.07.107

来源: Elsevier

【 摘 要 】

Despite significant progress toward super resolving more realistic images by deeper convolutional neural networks (CNNs), reconstructing fine and natural textures still remains a challenging problem. Recent works on single image super resolution (SISR) are mostly based on optimizing pixel and content wise similarity between recovered and high-resolution (HR) images and do not benefit from recognizability of semantic classes. In this paper, we introduce a novel approach using categorical information to tackle the SISR problem; we present an encoder architecture able to extract and use semantic information to super-resolve a given image by using multitask learning, simultaneously for image super-resolution and semantic segmentation. To explore categorical information during training, the proposed decoder only employs one shared deep network for two task-specific output layers. At run-time only layers resulting HR image are used and no segmentation label is required. Extensive perceptual experiments and a user study on images randomly selected from COCO-Stuff dataset demonstrate the effectiveness of our proposed method and it outperforms the state-of-the-art methods. (C) 2019 Elsevier B.V. All rights reserved.

【 授权许可】

Free