Multi-modal sensing and analysis of poster conversations with smart posterboard

APSIPA Transactions on Signal and Information Processing

Takuma Iwatate¹  Koji Inoue¹  Hiromasa Yoshimoto¹  Tatsuya Kawahara¹  Soichiro Hayashi¹  Katsuya Takanashi¹ 

[1] Kyoto University

关键词: Multi-modal signal processing;    Conversation analysis;    Behavioral analysis;    Speaker diarization;   

DOI  :  10.1017/ATSIP.2016.2

学科分类：计算机科学（综合）

来源: Cambridge University Press

【 摘 要 】

Conversations in poster sessions in academic events, referred to as poster conversations, pose interesting, and challenging topics on multi-modal signal and information processing. We have developed a smart posterboard for multi-modal recording and analysis of poster conversations. The smart posterboard has multiple sensing devices to record poster conversations, so we can review who came to the poster and what kind of questions or comments he/she made. The conversation analysis incorporates face and eye-gaze tracking for effective speaker diarization. It is demonstrated that eye-gaze information is useful for predicting turn-taking and also improving speaker diarization. Moreover, high-level indexing of interest and comprehension level of the audience is explored based on the multi-modal behaviors during the conversation. This is realized by predicting the audience's speech acts such as questions and reactive tokens.

【 授权许可】

Unknown