Systematic Review of Multimodal Human–Computer Interaction

Informatics

Bedrich Benes¹  Alejandra J. Magana²  Julieta Noguez³  Sergio Ruiz³  José Martín Molina-Espinosa³  Jose Daniel Azofeifa³ 

[1] Department of Computer Science, Purdue University, West Lafayette, IN 47907, USA;Department of Computer and Information Technology, Purdue University, West Lafayette, IN 47907, USA;Department of Engineering and Sciences, Tecnologico de Monterrey, Monterrey 64849, NL, Mexico;

关键词: haptics;    virtual reality;    augmented reality;    human–computer interaction;    multimodal interaction;   

DOI  :  10.3390/informatics9010013

来源: DOAJ

【 摘 要 】

This document presents a systematic review of Multimodal Human–Computer Interaction. It shows how different types of interaction technologies (virtual reality (VR) and augmented reality, force and vibration feedback devices (haptics), and tracking) are used in different domains (concepts, medicine, physics, human factors/user experience design, transportation, cultural heritage, and industry). A systematic literature search was conducted identifying 406 articles initially. From these articles, we selected 112 research works that we consider most relevant for the content of this article. The articles were analyzed in-depth from the viewpoint of temporal patterns, frequency of usage in types of technology in different domains, and cluster analysis. The analysis allowed us to answer relevant questions in searching for the next steps in work related to multimodal HCI. We looked at the typical technology type, how the technology type and frequency have changed in time over each domain, and how papers are grouped across metrics given their similarities. This analysis determined that VR and haptics are the most widely used in all domains. While VR is the most used, haptic interaction is presented in an increasing number of applications, suggesting future work on applications that configure VR and haptic together.

【 授权许可】

Unknown