Where on the face do we look during phonemic restoration: An eye-tracking study

Frontiers in Psychology

Psychology

Luca Campanelli¹  Vanessa Harwood²  Joseph Molski²  Alisa Baron²  Daniel Kleinman³  Nicole Landi⁴  Julia Irwin⁵ 

[1] Department of Communicative Disorders, The University of Alabama, Tuscaloosa, AL, United States;Department of Communicative Disorders, University of Rhode Island, Kingston, RI, United States;Haskins Laboratories, New Haven, CT, United States;Haskins Laboratories, New Haven, CT, United States;Department of Psychological Sciences, University of Connecticut, Storrs, CT, United States;Haskins Laboratories, New Haven, CT, United States;Department of Psychology, Southern Connecticut State University, New Haven, CT, United States;

关键词: audiovisual integration;    phonemic restoration;    speech perception;    eye tracking;    gaze patterns;   

DOI  :  10.3389/fpsyg.2023.1005186

 received in 2022-07-28, accepted in 2023-04-28,  发布年份 2023

来源: Frontiers

【 摘 要 】

Face to face communication typically involves audio and visual components to the speech signal. To examine the effect of task demands on gaze patterns in response to a speaking face, adults participated in two eye-tracking experiments with an audiovisual (articulatory information from the mouth was visible) and a pixelated condition (articulatory information was not visible). Further, task demands were manipulated by having listeners respond in a passive (no response) or an active (button press response) context. The active experiment required participants to discriminate between speech stimuli and was designed to mimic environmental situations which require one to use visual information to disambiguate the speaker’s message, simulating different listening conditions in real-world settings. Stimuli included a clear exemplar of the syllable /ba/ and a second exemplar in which the formant initial consonant was reduced creating an /a/−like consonant. Consistent with our hypothesis, results revealed that the greatest fixations to the mouth were present in the audiovisual active experiment and visual articulatory information led to a phonemic restoration effect for the /a/ speech token. In the pixelated condition, participants fixated on the eyes, and discrimination of the deviant token within the active experiment was significantly greater than the audiovisual condition. These results suggest that when required to disambiguate changes in speech, adults may look to the mouth for additional cues to support processing when it is available.

【 授权许可】

Unknown   
Copyright © 2023 Baron, Harwood, Kleinman, Campanelli, Molski, Landi and Irwin.