Learning a crowd-powered perceptual distance metric for facial blendshapes

EURASIP Journal on Image and Video Processing

Research

Zeynep Cipiloglu Yildiz¹ 

[1] Department of Computer Engineering, Manisa Celal Bayar University, Manisa, Turkey;

关键词: Blendshapes;    Animation;    Facial expressions;    Visual perception;    Crowdsourcing;    Metric learning;   

DOI  :  10.1186/s13640-023-00609-w

 received in 2023-01-26, accepted in 2023-05-09,  发布年份 2023

来源: Springer

【 摘 要 】

It is known that purely geometric distance metrics cannot reflect the human perception of facial expressions. A novel perceptually based distance metric designed for 3D facial blendshape models is proposed in this paper. To develop this metric, comparative evaluations of facial expressions were collected from a crowdsourcing experiment. Then, the weights of a distance metric, based on descriptive features of the models, were optimized to match the results with crowdsourced data, through a metric learning process. The method incorporates perceptual properties such as curvature and visual saliency. A formal analysis of the results proves the high correlation between the metric output and human perception. The effectiveness and success of the proposed metric were also compared to other distance alternatives. The proposed metric will enable intelligent processing of 3D facial blendshapes data in several ways. It will be possible to generate perceptually valid clustering and visualization of 3D facial blendshapes. It will help reduce storage and computational requirements by removing redundant expressions that are perceptually identical from the overall dataset. It can also be used to assist novice animators while creating plausible and expressive facial animations.

【 授权许可】

CC BY   
© The Author(s) 2023

【 图 表 】

40517_2023_258_Article_IEq122.gif

40517_2023_258_Article_IEq121.gif

40517_2023_258_Article_IEq119.gif

Fig. 1

40517_2023_258_Article_IEq115.gif

40517_2023_258_Article_IEq114.gif

40517_2023_258_Article_IEq113.gif

Fig. 1

Fig. 1

40517_2023_256_Article_IEq14.gif

Fig. 8

Fig. 6

Fig. 7

40517_2023_256_Article_IEq11.gif

Fig. 5

40517_2023_256_Article_IEq5.gif

40517_2023_256_Article_IEq4.gif

41116_2023_36_Article_IEq117.gif

Fig. 2

41116_2023_36_Article_IEq116.gif

41116_2023_36_Article_IEq115.gif

41116_2023_36_Article_IEq818.gif

41116_2023_36_Article_IEq817.gif

41116_2023_36_Article_IEq816.gif

41116_2023_36_Article_IEq815.gif

41116_2023_36_Article_IEq813.gif

41116_2023_36_Article_IEq811.gif

41116_2023_36_Article_IEq809.gif

40517_2023_258_Article_IEq102.gif

41116_2023_36_Article_IEq805.gif

41116_2023_36_Article_IEq803.gif

41116_2023_36_Article_IEq795.gif

41116_2023_36_Article_IEq789.gif

41116_2023_36_Article_IEq781.gif

Fig. 2

41116_2023_36_Article_IEq765.gif

41116_2023_36_Article_IEq753.gif

41116_2023_36_Article_IEq747.gif

41116_2023_36_Article_IEq744.gif

41116_2023_36_Article_IEq733.gif

Fig. 1

Fig. 6

40517_2023_258_Article_IEq51.gif

41116_2023_36_Article_IEq718.gif

40517_2023_258_Article_IEq46.gif