SemNet: Learning semantic attributes for human activity recognition with deep belief networks

Frontiers in Big Data

Big Data

Ole J. Mengshoel¹  Cathy Shunwen Tan²  John Paul Shen³  Shanmuga Venkatachalam³  Ming Zeng³  Harideep Nair³ 

[1] Department of Computer Science, Norwegian University of Science and Technology (NTNU), Trondheim, Norway;Department of ECE, Anderson School of Management, University of California, Los Angeles, Los Angeles, CA, United States;Department of ECE, Carnegie Mellon University, Pittsburgh, PA, United States;

关键词: human activity recognition;    deep belief networks;    semantic mid-level features;    ubiquitous computing;    multimodal sensing;    artificial intelligence;    internet of things;   

DOI  :  10.3389/fdata.2022.879389

 received in 2022-02-19, accepted in 2022-08-08,  发布年份 2022

来源: Frontiers

【 摘 要 】

Human Activity Recognition (HAR) is a prominent application in mobile computing and Internet of Things (IoT) that aims to detect human activities based on multimodal sensor signals generated as a result of diverse body movements. Human physical activities are typically composed of simple actions (such as “arm up”, “arm down”, “arm curl”, etc.), referred to as semantic features. Such abstract semantic features, in contrast to high-level activities (“walking”, “sitting”, etc.) and low-level signals (raw sensor readings), can be developed manually to assist activity recognition. Although effective, this manual approach relies heavily on human domain expertise and is not scalable. In this paper, we address this limitation by proposing a machine learning method, SemNet, based on deep belief networks. SemNet automatically constructs semantic features representative of the axial bodily movements. Experimental results show that SemNet outperforms baseline approaches and is capable of learning features that highly correlate with manually defined semantic attributes. Furthermore, our experiments using a different model, namely deep convolutional LSTM, on household activities illustrate the broader applicability of semantic attribute interpretation to diverse deep neural network approaches. These empirical results not only demonstrate that such a deep learning technique is semantically meaningful and superior to its handcrafted counterpart, but also provides a better understanding of the deep learning methods that are used for Human Activity Recognition.

【 授权许可】

Unknown   
Copyright © 2022 Venkatachalam, Nair, Zeng, Tan, Mengshoel and Shen.