Mask-aware photorealistic facial attribute manipulation

Computational Visual Media

Chen Huang¹  Hengliang Zhu²  Lizhuang Ma²  Ruoqi Sun² 

[1] Robotics Institute, Carnegie Mellon University, 15213, Pittsburgh, PA, USA;Shanghai Jiao Tong University, 200240, Shanghai, China;

关键词: face attribute manipulation;    generative adversarial network (GAN);    variational autoencoder (VAE);    partial dilated layers;    photorealism;   

DOI  :  10.1007/s41095-021-0219-7

来源: Springer

【 摘 要 】

The technique of facial attribute manipulation has found increasing application, but it remains challenging to restrict editing of attributes so that a face’s unique details are preserved. In this paper, we introduce our method, which we call a mask-adversarial autoencoder (M-AAE). It combines a variational autoencoder (VAE) and a generative adversarial network (GAN) for photorealistic image generation. We use partial dilated layers to modify a few pixels in the feature maps of an encoder, changing the attribute strength continuously without hindering global information. Our training objectives for the VAE and GAN are reinforced by supervision of face recognition loss and cycle consistency loss, to faithfully preserve facial details. Moreover, we generate facial masks to enforce background consistency, which allows our training to focus on the foreground face rather than the background. Experimental results demonstrate that our method can generate high-quality images with varying attributes, and outperforms existing methods in detail preservation.

【 授权许可】

CC BY