Human and machine recognition of dynamic and static facial expressions: prototypicality, ambiguity, and complexity

Frontiers in Psychology

Psychology

Dennis Küster¹  Jeffrey M. Girard²  Hyunwoo Kim³  Eva G. Krumhuber³ 

[1]Cognitive Systems Lab, Department of Mathematics and Computer Science, University of Bremen, Bremen, Germany

[2]Department of Psychology, University of Kansas, Lawrence, KS, United States

[3]Departmet of Experimental Psychology, University College London, London, United Kingdom

关键词: emotion facial expression;    dynamic;    movement;    prototypicality;    ambiguity;   

DOI  :  10.3389/fpsyg.2023.1221081

 received in 2023-05-11, accepted in 2023-08-22,  发布年份 2023

来源: Frontiers

【 摘 要 】

A growing body of research suggests that movement aids facial expression recognition. However, less is known about the conditions under which the dynamic advantage occurs. The aim of this research was to test emotion recognition in static and dynamic facial expressions, thereby exploring the role of three featural parameters (prototypicality, ambiguity, and complexity) in human and machine analysis. In two studies, facial expression videos and corresponding images depicting the peak of the target and non-target emotion were presented to human observers and the machine classifier (FACET). Results revealed higher recognition rates for dynamic stimuli compared to non-target images. Such benefit disappeared in the context of target-emotion images which were similarly well (or even better) recognised than videos, and more prototypical, less ambiguous, and more complex in appearance than non-target images. While prototypicality and ambiguity exerted more predictive power in machine performance, complexity was more indicative of human emotion recognition. Interestingly, recognition performance by the machine was found to be superior to humans for both target and non-target images. Together, the findings point towards a compensatory role of dynamic information, particularly when static-based stimuli lack relevant features of the target emotion. Implications for research using automatic facial expression analysis (AFEA) are discussed.

【 授权许可】

Unknown   
Copyright © 2023 Kim, Küster, Girard and Krumhuber.