A self-organizing neural network architecture for learning human-object interactions

NEUROCOMPUTING

卷:307

Article

Mici, Luiza¹  Parisi, German I.¹  Wermter, Stefan¹ 

[1] Univ Hamburg, Dept Informat, Knowledge Technol, Hamburg, Germany

关键词: Self-organization;    Hierarchical learning;    Action recognition;    Object recognition;    Human-object interaction;   

DOI  :  10.1016/j.neucom.2018.04.015

来源: Elsevier

【 摘 要 】

The visual recognition of transitive actions comprising human-object interactions is a key component for artificial systems operating in natural environments. This challenging task requires jointly the recognition of articulated body actions as well as the extraction of semantic elements from the scene such as the identity of the manipulated objects. In this paper, we present a self-organizing neural network for the recognition of human-object interactions from RGB-D videos. Our model consists of a hierarchy of Grow-When-Required (GWR) networks that learn prototypical representations of body motion patterns and objects, accounting for the development of action-object mappings in an unsupervised fashion. We report experimental results on a dataset of daily activities collected for the purpose of this study as well as on a publicly available benchmark dataset. In line with neurophysiological studies, our self-organizing architecture exhibits higher neural activation for congruent action-object pairs learned during training sessions with respect to synthetically created incongruent ones. We show that our unsupervised model shows competitive classification results on the benchmark dataset with respect to strictly supervised approaches. (C) 2018 The Authors. Published by Elsevier B.V.

【 授权许可】

Free