Robust Facial Expression Recognition Based on Local Directional Pattern

ETRI Journal

关键词: support vector machine;    principal component analysis;    features extraction;    local directional pattern;    facial expression recognition;    Image representation;   

Others  :  1185901
DOI  :  10.4218/etrij.10.1510.0132

【 摘 要 】

Automatic facial expression recognition has many potential applications in different areas of human computer interaction. However, they are not yet fully realized due to the lack of an effective facial feature descriptor. In this paper, we present a new appearance-based feature descriptor, the local directional pattern (LDP), to represent facial geometry and analyze its performance in expression recognition. An LDP feature is obtained by computing the edge response values in 8 directions at each pixel and encoding them into an 8 bit binary number using the relative strength of these edge responses. The LDP descriptor, a distribution of LDP codes within an image or image patch, is used to describe each expression image. The effectiveness of dimensionality reduction techniques, such as principal component analysis and AdaBoost, is also analyzed in terms of computational cost saving and classification accuracy. Two well-known machine learning methods, template matching and support vector machine, are used for classification using the Cohn-Kanade and Japanese female facial expression databases. Better classification accuracy shows the superiority of LDP descriptor against other appearance-based feature descriptors.

   

【 参考文献 】

• [1]Y.L. Tian et al., "Real World Real-time Automatic Recognition of Facial Expressions," Proc. IEEE Workshop Performance Evaluation of Tracking and Surveillance, 2003.
• [2]C. Shan, S. Gong, and P.W. McOwan, "Robust Facial Expression Recognition using Local Binary Patterns," Proc. IEEE Int. Conf. Image Process., 2005, pp. 914-917.
• [3]M.Z. Uddin, J.J. Lee, and T.S. Kim, "An Enhanced Independent Component-Based Human Facial Expression Recognition from Video," IEEE Trans. Consum. Electron., vol. 55, no. 4, 2009, pp. 2216-2224.
• [4]M.C. Hwang et al., "Person Identification System for Future Digital TV with Intelligence," IEEE Trans. Consum. Electron., vol. 53, no. 1, 2007, pp. 218-226.
• [5]P. Corcoran et al., "Biometric Access Control for Digital Media Streams in Home Networks," IEEE Trans. Consum. Electron., vol. 53, no. 3, 2007, pp. 917-925.
• [6]M. Pantic and L.J.M. Rothkrantz, "Automatic Analysis of Facial Expressions: The State of the Art," IEEE Trans. Pattern Anal. Mach. Intell., vol. 22, no. 12, 2000, pp. 1424-1445.
• [7]B. Fasel and J. Luettin, "Automatic Facial Expression Analysis: A Survey," Pattern Recog., vol. 36, no. 1, 2003, pp. 259-275.
• [8]G. Zhao and M. Pietikainen, "Boosted Multi-resolution Spatio-Temporal Descriptors for Facial Expression Recognition," Pattern Recognit. Lett., vol. 30, no. 12, 2009, pp. 1117-1127.
• [9]C. Shan, S. Gong, and P.W. McOwan, "Facial Expression Recognition based on Local Binary Patterns: A Comprehensive Study," Image Vision Comput., vol. 27, no. 6, May 2009, pp. 803-816.
• [10]K.H. Choi et al., "A Probabilistic Network for Facial Feature Verification," ETRI J., vol. 25, no. 2, Apr. 2003, pp. 140-143.
• [11]K.H. Kim et al., "Facial Feature Extraction Based on Private Energy Map in DCT Domain," ETRI J., vol. 29, no. 2, Apr. 2007, pp. 243-245.
• [12]Y. Tian, T. Kanade, and J.F. Cohn, "Facial Expression Analysis," Handbook of Face Recognition, Springer, Oct. 2003.
• [13]Z. Zhang et al., "Comparison between Geometry-Based and Gabor-wavelets-based Facial Expression Recognition Using Multi-layer Perceptron," Proc. IEEE Int. Conf. Auto. Face Gesture Recog., Apr. 1998, pp. 454-459.
• [14]P. Ekman and W. Friesen, Facial Action Coding System: A Technique for Measurement of Facial Movement, Consulting Psychologists Press, 1978.
• [15]M. Valstar, I. Patras, and M. Pantic, "Facial Action Unit Detection using Probabilistic Actively Learned Support Vector Machines on Tracked Facial Point Data," IEEE CVPR Workshop, vol. 3, 2005, pp. 76-84.
• [16]M. Valstar and M. Pantic, "Fully Automatic Facial Action Unit Detection and Temporal Analysis," IEEE CVPR Workshop, June 2006, p. 149.
• [17]M.A. Turk and A.P. Pentland, "Face Recognition Using Eigenfaces," Proc. Comput. Vision Pattern Recog., 1991, pp. 586-591.
• [18]C. Padgett and G. Cottrell, "Representation Face Images for Emotion Classification," Advances Neural Inf. Process. Syst., vol. 9, Cambridge, MA, MIT Press, 1997.
• [19]M.S. Bartlett, J.R. Movellan, and T.J. Sejnowski, "Face Recognition by Independent Component Analysis," IEEE Trans. Neural Networks, vol. 13, no. 6, 2002, pp. 1450-1464.
• [20]C.C. Fa and F.Y. Shin, "Recognizing Facial Action Units using Independent Component Analysis and Support Vector Machine," Pattern Recog., vol. 39, no. 9, 2006, pp. 1795-1798.
• [21]M.J. Lyons, J. Budynek, and S. Akamatsu, "Automatic Classification of Single Facial Images," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 12, 1999, pp. 1357-1362.
• [22]G. Donato et al., "Classifying Facial Actions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 21, no. 10, 1999, pp. 974-989.
• [23]T. Ojala and M. Pietikainen, "Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns," IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, 2002, pp. 971-987.
• [24]X. Feng, M. Pietikainen, and A. Hadid, "Facial Expression Recognition with Local Binary Patterns and Linear Programming," Pattern Recog. Image Anal., vol. 15, no. 2, 2005, pp. 546-548.
• [25]G. Zhao and M. Pietikainen, "Dynamic Texture Recognition using Local Binary Patterns with An Application to Facial Expressions," IEEE Trans. Pattern Anal. Mach. Intell., vol. 29, no. 6, 2007, pp. 915-928.
• [26]H. Zhou, R. Wang, and C. Wang, "A Novel Extended Local Binary Pattern Operator for Texture Analysis," Inf. Science, vol. 178, no. 22, 2008, pp. 4314-4325.
• [27]T. Jabid, M.H. Kabir, and O.S. Chae, "Local Directional Pattern (LDP) for Face Recognition," IEEE Int. Conf. Consum. Electron., 2010, pp. 329-330.
• [28]T. Kanade, J. Cohn, and Y. Tian, "Comprehensive Database for Facial Expression Analysis," IEEE Int. Conf. Autom. Face Gesture Recog., Mar. 2000, pp. 46-53.
• [29]S. Zhao, Y. Gao, and B. Zhang, "Sobel-LBP," Int. Conf. Image Process., 2008, pp. 2144-2147.
• [30]R. Mattivi and L. Shao, "Human Action Recognition Using LBP-TOP as Sparse Spatio-Temporal Feature Descriptor," Int. Conf. Comput. Anal. Image Pattern, 2009, pp. 740-747.
• [31]W.K. Pratt, Digital Image Processing, Wiley, New York, 1978.
• [32]S.W. Lee, "Off-line Recognition of Totally Unconstrained Handwritten Numerals Using Multilayer Cluster Neural Network," IEEE Trans. Pattern Anal. Mach. Intell., vol. 18, no. 6, 1996, pp. 648-652.
• [33]T. Jabid, M.H. Kabir, and O.S. Chae, "Local Directional Pattern (LDP): A Robust Image Descriptor for Object Recognition," Accepted in Adv. Video and Signal-Based Surveillance, 2010.
• [34]D. Lowe, "Distinctive Image Features from Scale Invariant Key Points," Int. J. Comput. Vision, vol. 60, no. 2, 2004, pp. 91-110.
• [35]T. Ahonen, A. Hadid, and M. Pietikainen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Trans. Pattern Anal. Mach. Intell., vol. 28, no. 12, 2006, pp. 2037-2041.
• [36]S. Gundimada and V.K. Asari, "Facial Recognition Using Multisensor Images Based on Localized Kernel Eigen Spaces," IEEE Trans. Image Process., vol. 18, no. 6, 2009, pp. 1314-1325.
• [37]C.A. Kumar, "Analysis of Unsupervised Dimensionality Reduction Techniques," Comput. Sci. Inf. Syst., vol. 6, no. 2, Dec. 2009, pp. 217-227.
• [38]Y. Freund and R.E. Schapire, "A Decision-Theoretic Generalization of On-line Learning and an Application to Boosting," Computational Learning Theory, 1995, pp. 23-37.
• [39]R.E. Schapire and Y. Singer, "Improved Boosting Algorithms using Confidence-Rated Predictions," Maching Learning, vol. 37, no.3, 1999, pp. 297-336.
• [40]C. Cortes and V. Vapnik, "Support Vector Networks," Machine Learning, vol. 20, no. 3, 1995, pp. 273-297.
• [41]C.W. Hsu and C.J. Lin, "A Comparison on Methods for Multiclass Support Vector Machines," IEEE Trans. Neural Networks, vol. 13, no. 2, 2002, pp. 415-425.
• [42]Z. Niu et al., "2D Cascaded AdaBoost for Eye Localization," Proc. IEEE Int. Conf. Pattern Recog., 2006, pp. 1216-1219.
• [43]M.S. Bartlett et al., "Recognizing Facial Expression: Machine Learning and Application to Spontaneous Behavior," IEEE Conf. Computer Vision and Pattern Recog., 2005, pp. 568-573.
• [44]Y. Tian, "Evaluation of Face Resolution for Expression Analysis," CVPR Workshop Face Process. Video, 2004, p. 82.
• [45]Y. Ono, FT. Okabe, and Y. Sato, "Gaze Estimation from Low Resolution Images," IEEE Pacific-Rim Symp. Image Video Technol., 2006, pp. 178-188.
• [46]J.N. Bassili, "Emotion Recognition: The Role of Facial Movement and the Relative Importance of Upper and Lower Areas of the Face," J. Personality Social Psychology, vol. 37, no. 11, 1979, pp. 2049-2058.