Multimodality matters in numerical communication

Frontiers in Psychology

Psychology

Tyler Marghetis¹  Bodo Winter² 

[1] Cognitive and Information Sciences, University of California, Merced, Merced, CA, United States;Department of English Language and Linguistics, University of Birmingham, Birmingham, United Kingdom;

关键词: numerical cognition;    numerical communication;    multimodality;    data visualization;    graphs;    gesture;    quantifiers;   

DOI  :  10.3389/fpsyg.2023.1130777

 received in 2022-12-23, accepted in 2023-05-10,  发布年份 2023

来源: Frontiers

【 摘 要 】

Modern society depends on numerical information, which must be communicated accurately and effectively. Numerical communication is accomplished in different modalities—speech, writing, sign, gesture, graphs, and in naturally occurring settings it almost always involves more than one modality at once. Yet the modalities of numerical communication are often studied in isolation. Here we argue that, to understand and improve numerical communication, we must take seriously this multimodality. We first discuss each modality on its own terms, identifying their commonalities and differences. We then argue that numerical communication is shaped critically by interactions among modalities. We boil down these interactions to four types: one modality can amplify the message of another; it can direct attention to content from another modality (e.g., using a gesture to guide attention to a relevant aspect of a graph); it can explain another modality (e.g., verbally explaining the meaning of an axis in a graph); and it can reinterpret a modality (e.g., framing an upwards-oriented trend as a bad outcome). We conclude by discussing how a focus on multimodality raises entirely new research questions about numerical communication.

【 授权许可】

Unknown   
Copyright © 2023 Winter and Marghetis.