Study on Automatic 3D Facial Caricaturization: From Rules to Deep Learning

Frontiers in Virtual Reality

Virtual Reality

Ludovic Hoyet¹  Ferran Arguelaguet¹  Franck Multon²  Quentin Avril³  Philippe Guillotel³  Fabien Danieau³  Nicolas Olivier⁴  Glenn Kerbiriou⁵ 

[1] Inria, Univ Rennes, CNRS, IRISA, Rennes, France;Inria, Univ Rennes, CNRS, IRISA, Rennes, France;Laboratoire Mouvement, Sport, Santé, Bruz, France;InterDigital, Cesson-Sévigné, France;InterDigital, Cesson-Sévigné, France;Inria, Univ Rennes, CNRS, IRISA, Rennes, France;InterDigital, Cesson-Sévigné, France;Institut national des sciences appliquées de Rennes, Rennes, France;

关键词: caricature;    style transfer;    machine learning;    geometry processing;    3D mesh;    perceptual study;   

DOI  :  10.3389/frvir.2021.785104

 received in 2021-09-28, accepted in 2021-12-06,  发布年份 2022

来源: Frontiers

【 摘 要 】

Facial caricature is the art of drawing faces in an exaggerated way to convey emotions such as humor or sarcasm. Automatic caricaturization has been explored both in the 2D and 3D domain. In this paper, we propose two novel approaches to automatically caricaturize input facial scans, filling gaps in the literature in terms of user-control, caricature style transfer, and exploring the use of deep learning for 3D mesh caricaturization. The first approach is a gradient-based differential deformation approach with data driven stylization. It is a combination of two deformation processes: facial curvature and proportions exaggeration. The second approach is a GAN for unpaired face-scan-to-3D-caricature translation. We leverage existing facial and caricature datasets, along with recent domain-to-domain translation methods and 3D convolutional operators, to learn to caricaturize 3D facial scans in an unsupervised way. To evaluate and compare these two novel approaches with the state of the art, we conducted the first user study of facial mesh caricaturization techniques, with 49 participants. It highlights the subjectivity of the caricature perception and the complementarity of the methods. Finally, we provide insights for automatically generating caricaturized 3D facial mesh.

【 授权许可】

Unknown   
Copyright © 2022 Olivier, Kerbiriou, Arguelaguet, Avril, Danieau, Guillotel, Hoyet and Multon.