Do less and achieve more: Training CNNs for action recognition utilizing action images from the Web

PATTERN RECOGNITION

卷:68

Article

Ma, Shugao¹  Bargal, Sarah Adel¹  Zhang, Jianming³  Sigal, Leonid²  Sclaroff, Stan¹ 

[1] Boston Univ, 111 Cummington Mall, Boston, MA 02215 USA

[2] Disney Res, 4720 Forbes Ave, Pittsburgh, PA 15213 USA

[3] Adobe Res, 345 Pk Ave, San Jose, CA 95110 USA

关键词: Action recognition;    Convolutional neural networks;    Deep learning;    Videos;    Images;   

DOI  :  10.1016/j.patcog.2017.01.027

来源: Elsevier

【 摘 要 】

Recently, attempts have been made to collect millions of videos to train Convolutional Neural Network (CNN) models for action recognition in videos. However, curating such large-scale video datasets requires immense human labor, and training CNNs on millions of videos demands huge computational resources. In contrast, collecting action images from the Web is much easier and training on images requires much less computation. In addition, labeled web images tend to contain discriminative action poses, which highlight discriminative portions of a video's temporal progression. Through extensive experiments, we explore the question of whether we can utilize web action images to train better CNN models for action recognition in videos. We collect 23.8K manually filtered images from the Web that depict the 101 actions in the UCF101 action video dataset. We show that by utilizing web action images along with videos in training, significant performance boosts of CNN models can be achieved. We also investigate the scalability of the process by leveraging crawled web images (unfiltered) for UCF101 and ActivityNet. Using unfiltered images we can achieve performance improvements that are on-par with using filtered images. This means we can further reduce annotation labor and easily scale-up to larger problems. We also shed light on an artifact of finetuning CNN models that reduces the effective parameters of the CNN and show that using web action images can significantly alleviate this problem. (C) 2017 Elsevier Ltd. All rights reserved.

【 授权许可】

Free