An end-to-end framework for unconstrained monocular 3D hand pose estimation

PATTERN RECOGNITION

卷:115

Article

Sharma, Sanjeev¹  Huang, Shaoli¹ 

[1] Univ Sydney, J12 Cleveland St, Darlington, NSW 2008, Australia

关键词: Hand detection;    Hand tracking;    Hand pose estimation;    Computer vision;    Deep learning;   

DOI  :  10.1016/j.patcog.2021.107892

来源: Elsevier

【 摘 要 】

This work addresses the challenging problem of unconstrained 3D hand pose estimation using monocular RGB images. Most of the existing approaches assume some prior knowledge of hand (such as hand locations and side information) is available for 3D hand pose estimation. This restricts their use in unconstrained environments. Therefore, we present an end-to-end framework that robustly predicts hand prior information and accurately infers 3D hand pose by learning ConvNet models while only using keypoint annotations. To enhance the hand detector's robustness, we propose a novel keypoint-based method to simultaneously predict hand regions and side labels, unlike existing methods that suffer from background color confusion caused by using segmentation or detection-based technology. Moreover, inspired by the human hand's biological structure, we introduce two geometric constraints directly into the 3D coordinates prediction that further improves its performance. Experimental results show that our proposed framework outperforms the state-of-art methods on standard benchmark datasets while providing robust predictions. Crown Copyright (c) 2021 Published by Elsevier Ltd. All rights reserved.

【 授权许可】

Free