Topic modelling for routine discovery from egocentric photo-streams

PATTERN RECOGNITION

卷:104

Article

Talavera, Estefania^1,2,3  Wuerich, Carolin^2,3  Petkov, Nicolai¹  Radeva, Petia^2,3 

[1] Univ Groningen, Johann Bernoulli Inst, Nijenborgh 9, NL-9747 AG Groningen, Netherlands

[2] Univ Barcelona, Dept Math & Comp Sci, Gran Via Corts Catalanes 585, Barcelona 08007, Spain

[3] Univ Barcelona, Comp Vis Ctr, Gran Via Corts Catalanes 585, Barcelona 08007, Spain

关键词: Routine;    Egocentric vision;    Lifestyle;    Behaviour analysis;    Topic modelling;   

DOI  :  10.1016/j.patcog.2020.107330

来源: Elsevier

【 摘 要 】

Developing tools to understand and visualize lifestyle is of high interest when addressing the improvement of habits and well-being of people. Routine, defined as the usual things that a person does daily, helps describe the individuals' lifestyle. With this paper, we are the first ones to address the development of novel tools for automatic discovery of routine days of an individual from his/her egocentric images. In the proposed model, sequences of images are firstly characterized by semantic labels detected by pre-trained CNNs. Then, these features are organized in temporal-semantic documents to later be embedded into a topic models space. Finally, Dynamic-Time-Warping and Spectral-Clustering methods are used for final day routine/non-routine discrimination. Moreover, we introduce a new EgoRoutine-dataset, a collection of 104 egocentric days with more than 100.000 images recorded by 7 users. Results show that routine can be discovered and behavioural patterns can be observed. (C) 2020 The Author(s). Published by Elsevier Ltd.

【 授权许可】

Free