In this dissertation we study core problems in 3D face processing with their important applications in biometrics and graphics. We propose efficient and accurate feature based algorithms for facial feature detection, 3D model fitting and tracking from static and dynamic 2D and 3D visual signal input. We also demonstrate the applications of 3D face modeling in avatar animation, expression recognition and eye gaze estimation. The algorithms contribute to a complete framework for image and video analysis and synthesis based on 3D face models. From the experiments on the self-collected and public database, we argue that feature based 3D face processing is robust to variations of environment, competent for highly accurate reconstruction of 3D facial geometry and motion, and suitable for realtime biometrics and animation applications.The contributions of this dissertation include: (1) A novel interactive robust facial component localization algorithm designed to accommodate partial labels on natural images. (2) A new challenging dataset for facial component localization which contains 2330 high-resolution, accurately labeled face images and has larger degree of out-of-plane orientation and occlusion typical of real-world scenarios. (3) Efficient algorithms for fitting and tracking of 3D model to 2D videos and RGB-D videos based on an iterative linearized optimization procedure. (4) An MPEG-4 compatible performance driven avatar capable of rendering of highly detail shape and texture of the subject in real time. (5) A new method for measuring similarity of 3D facial shapes using facial level curves. The measurement comprises the spatio-temporal features for expression recognition from 3D dynamic faces through a hidden Markov model. (6) An efficient and accurate system for measuring the online student's point of gaze using commodity webcams.
PDF
download
878987797
028-85220240
