Vision-Based Finger Action Recognition by Angle Detection and Contour Analysis

ETRI Journal

关键词: machine vision;    human-machine interaction;    scale-invariant angle detection;    fingertip tracking;    Fingertip detection;   

Others  :  1186112
DOI  :  10.4218/etrij.11.0110.0313

【 摘 要 】

In this paper, we present a novel vision-based method of recognizing finger actions for use in electronic appliance interfaces. Human skin is first detected by color and consecutive motion information. Then, fingertips are detected by a novel scale-invariant angle detection based on a variable k-cosine. Fingertip tracking is implemented by detected region-based tracking. By analyzing the contour of the tracked fingertip, fingertip parameters, such as position, thickness, and direction, are calculated. Finger actions, such as moving, clicking, and pointing, are recognized by analyzing these fingertip parameters. Experimental results show that the proposed angle detection can correctly detect fingertips, and that the recognized actions can be used for the interface with electronic appliances.

   

【 参考文献 】

• [1]G. Hillebrand and K. Zuerl, "Finger Tracking for Virtual Agents," LNCS, vol. 4722, 2007, pp. 417-419.
• [2]G. Hillebrand et al., "Inverse Kinematic Infrared Optical Finger Tracking," Int. Conf. Humans and Computers, 2006.
• [3]P.G. Kry et al., "HandNavigator: Hands-On Interaction for Desktop Virtual Reality," ACM Symp. Virtual Reality Software Technol., Oct. 2008, pp. 53-60.
• [4]R.Y. Wang and J. Popovic, "Real-Time Hand-Tracking with a Color Glove," Int. Conf. Computer Graphics Interactive Techn., 2009.
• [5]S.H. Lee, J. Choi, and J. Park, "Interactive e-Learning System using Pattern Recognition and Augmented Reality," IEEE Trans. Consumer Electron., vol. 55, no. 2, May, 2009, pp. 883-890.
• [6]F. Duca, J. Fredriksson, and M. Fjeld, "Real-Time 3D Hand Interaction: Single Webcam Low-Cost Approach," IEEE Workshop Trends Issues Tracking Virtual Environments, 2007, pp. 1-5.
• [7]J. Segen and S. Kumar, "Look Ma, No Mouse!" Commun. ACM, vol. 43, no. 7, July 2000, pp. 102-109.
• [8]J. Segen and S. Kumar, "GestureVR: Vision-Based 3D Hand Interface for Spatial Interaction," Proc. 6th ACM Int. Conf. Multimedia, 1998, pp. 455-464.
• [9]J.M. Kim and W.K. Lee, "Hand Shape Recognition Using Fingertips," 5th Int. Conf. Fuzzy Syst. Knowledge Discovery, 2008, pp. 44-48.
• [10]D. Lee and Y. Park, "Vision-Based Remote Control System by Motion Detection and Open Finger Counting," IEEE Trans. Consumer Electron., vol. 55, no. 4, Nov. 2009, pp. 2308-2313.
• [11]K. Oka, Y. Sato, and H. Koike, "Real-Time Fingertip Tracking and Gesture Recognition," IEEE Comp. Graph. Appl., vol. 22, no. 6, Dec. 2002, pp. 64-71.
• [12]S.C. Crampton and M. Betke, "Counting Fingers in Real Time: A Webcam-Based Human-Computer Interface with Game Applications," Proc. Conf. Universal Access Human-Comput. Interaction, 2003, pp. 1357-1361.
• [13]J. MacLean et al., "Fast Hand Gesture Recognition for Real-Time Teleconferencing Applications," Int. Workshop Recognition, Analysis Tracking of Faces Gestures Real Time Syst., 2001, pp. 133-140.
• [14]R. Belaroussi and M. Milgram, "A Real Time Fingers Detection by Symmetry Transform Using a Two Cameras System," LNCS, vol. 5359, 2008, pp. 703-712.
• [15]O. Gallo, S.M. Arteaga, and J.E. Davis, "A Camera-Based Pointing Interface for Mobile Devices," IEEE Int. Conf. Image Processing, 2008, pp. 1420-1423.
• [16]B. Stenger et al., "Model-Based Hand Tracking Using a Hierarchical Bayesian Filter," IEEE Trans. Patt. Anal. Mach. Intell., vol. 28, no. 9, Sept. 2006, pp. 1372-1384.
• [17]J. Alon et al., "A Unified Framework for Gesture Recognition and Spatiotemporal Gesture Segmentation," IEEE Trans. Patt. Anal. Mach. Intell., vol. 31, no. 9, Sept. 2009, pp. 1685-1699.
• [18]H. Ying et al., "Fingertip Detection and Tracking Using 2D and 3D Information," Proc. 7th World Congress Intell. Control Autom., 2008, pp. 1149-1152.
• [19]C. Nolker and H. Ritter, "Detection of Fingertips in Human Hand Movement Sequences," Gesture Sign Language Human-Computer Interaction, 1997, pp. 209-218.
• [20]A.M. Burns and B. Mazzarino. "Finger Tracking Methods Using EyesWeb," LNCS, vol. 3881, 2006, pp. 156-167.
• [21]J. Hannuksela et al., "Motion-Based Finger Tracking for User Interface with Mobile Devices," IET 4th European Conf. Visual Media Production, 2007.
• [22]A.A. Argyros and M.I.A. Lourakis, "Vision-Based Interpretation of Hand Gestures for Remote Control of a Computer Mouse," LNCS, vol. 3979, 2006, pp. 40-51.
• [23]J. Hannuksela et al., "Adaptive Motion-Based Gesture Recognition Interface for Mobile Phones," LNCS, vol. 5008, 2008, pp. 271-280.
• [24]Y. Hirobe and T. Niikura, "Vision-Based Input Interface for Mobile Devices with High-Speed Fingertip Tracking," ACM Symp. User Interface, Software Technol., 2009, pp. 7-8.
• [25]Z. Zivkovic, "Improved Adaptive Gaussian Mixture Model for Background Subtraction," Int. Conf. Pattern Recog., 2004, pp. 28-31.
• [26]Y.J. Lee and D.H. Lee, "Research on Detecting Face and Hands for Motion-Based Game Using Web Camera," Int. Conf. Security Technol., 2008, pp. 7-12.
• [27]A. Rosenfeld and E. Johnston, "Angle Detection on Digital Curves," IEEE Trans. Comp., vol. 22, no. 9, Sept. 1973, pp. 875-878.
• [28]A. Rosenfeld and J.S. Weszka, "An Improved Method of Angle Detection on Digital Curves," IEEE Trans. Comp., vol. 24, no. 9, Sept. 1975, pp. 940-941.
• [29]R.E. Kalman, "A New Approach to Linear Filtering and Prediction Problems," J. Basic Eng., vol. 82-D, 1960, pp. 35-45.