Automatic role recognition

QA75 Electronic computers. Computer science

Salamin, Hugues Eric ; Vinciarelli, Alsessandro

University:University of Glasgow

Department:School of Computing Science

关键词: Social Signal Processing, Automatic Role Recognition;   

Others  :  http://theses.gla.ac.uk/4367/1/2013SalaminPhD.pdf

来源: University of Glasgow

【 摘 要 】

The computing community is making significant efforts towards the development of automatic approaches for the analysis of social interactions. The way people interact depends on the context, but there is one aspect that all social interactions seem to have in common: humans behave according to roles. Therefore, recognizing the roles of participants is an essential step towards understanding social interactions and the construction of socially awarecomputer.This thesis addresses the problem of automatically recognizing roles of participants in multi-party recordings. The objective is to assign to each participant a role. All the proposed approaches use a similar strategy. They all start by segmenting the audio into turns. Thoseturns are used as basic analysis units. The next step is to extract features accounting for the organization of turns. The more sophisticated approaches extend the features extracted with features from either the prosody or the semantic. Finally, the mapping of people or turns toroles is done using statistical models. The goal of this thesis is to gain a better understanding of role recognition and we will investigate three aspects that can influence the performanceof the system:We investigate the impact of modelling the dependency between the roles.We investigate the contribution of different modalities for the effectiveness of rolerecognition approach.We investigate the effectiveness of the approach for different scenarios.Three models are proposed and tested on three different corpora totalizing more than 90 hours of audio. The first contribution of this thesis is to investigate the combination of turn-taking features and semantic information for role recognition, improving the accuracy ofrole recognition from a baseline of 46.4% to 67.9% on the AMI meeting corpus. The second contribution is to use features extracted from the prosody to assign roles. The performance of this model is 89.7% on broadcast news and 87.0% on talk-shows. Finally, the third contribution is the development of a model robust to change in the social setting. This model achieved an accuracy of 86.7% on a database composed of a mixture of broadcast news andtalk-shows.