Metaphoric Gestures Facilitate Perception of Intonation More than Length in Auditory Judgments of Non-Native Phonemic Contrasts

Collabra: Psychology

Yukari Hirata¹  Spencer Kelly²  April Bailey³ 

[1] Department of East Asian Languages and Literatures, Center for Language and Brain, Colgate University;Department of Psychology, Center for Language and Brain, Colgate University;Department of Psychology, Yale University

关键词: foreign language;    speech;    hand gesture;    phoneme;    multimodal;    length contrasts;    intonational contrasts;    perception;   

DOI  :  10.1525/collabra.76

学科分类：社会科学、人文和艺术（综合）

来源: University of California Press

【 摘 要 】

It is well established that hand gestures affect comprehension and learning of semantic aspects of a foreign language (FL). However, much less is known about the role of hand gestures in lower-level language processes, such as perception of phonemes. To address this gap, we explored the role that metaphoric gestures play in perceiving FL speech sounds that varied on two dimensions: length and intonation. English speaking adults listened to Japanese length contrasts and sentence-final intonational distinctions in the context of congruent, incongruent and no gestures. For intonational contrasts, identification was more accurate for congruent gestures and less accurate for incongruent gestures relative to the baseline no gesture condition. However, for the length contrasts, there was no such clear and consistent pattern, and in fact, congruent gestures made speech processing more effortful. We conclude that metaphoric gestures help with some—but not all—novel speech sounds in a FL, suggesting that gesture and speech are phonemically integrated to differing extents depending on the nature of the gesture and/or speech sound. 

.

【 授权许可】

CC BY