Automatic landmark annotation and dense correspondence registration for 3D human facial images

BMC Bioinformatics

Jianya Guo¹  Xi Mei¹  Kun Tang¹ 

[1] CAS-MPG Partner Institute and Key Laboratory for Computational Biology, Shanghai Institutes for Biological Sciences, Chinese Academy of Sciences, Shanghai, China

关键词: Dense correspondence;    Landmark localization;    Registration;    Facial morphology;    3D face;   

Others  :  1087804
DOI  :  10.1186/1471-2105-14-232

 received in 2013-01-17, accepted in 2013-07-15,  发布年份 2013

【 摘 要 】

Background

Traditional anthropometric studies of human face rely on manual measurements of simple features, which are labor intensive and lack of full comprehensive inference. Dense surface registration of three-dimensional (3D) human facial images holds great potential for high throughput quantitative analyses of complex facial traits. However there is a lack of automatic high density registration method for 3D faical images. Furthermore, current approaches of landmark recognition require further improvement in accuracy to support anthropometric applications.

Result

Here we describe a novel non-rigid registration method for fully automatic 3D facial image mapping. This method comprises two steps: first, seventeen facial landmarks are automatically annotated, mainly via PCA-based feature recognition following 3D-to-2D data transformation. Second, an efficient thin-plate spline (TPS) protocol is used to establish the dense anatomical correspondence between facial images, under the guidance of the predefined landmarks. We demonstrate that this method is highly accurate in landmark recognition, with an average RMS error of ~1.7 mm. The registration process is highly robust, even for different ethnicities.

Conclusion

This method supports fully automatic registration of dense 3D facial images, with 17 landmarks annotated at greatly improved accuracy. A stand-alone software has been implemented to assist high-throughput high-content anthropometric analysis.

【 授权许可】

   
2013 Guo et al.; licensee BioMed Central Ltd.

【 图 表 】

Figure 1.

Figure 2.

Figure 3.

Figure 4.

Figure 5.

Figure 6.

【 参考文献 】

• [1]Ohya Y, Sese J, Yukawa M, Sano F, Nakatani Y, Saito TL, Saka A, Fukuda T, Ishihara S, Oka S: High-dimensional and large-scale phenotyping of yeast mutants. Proc Natl Acad Sci USA 2005, 102(52):19015.
• [2]Tobias C, Roberto H, Falk S: HTPheno: an image analysis pipeline for high-throughput plant phenotyping. BMC Bioinforma 2011, 12(1):148. BioMed Central Full Text
• [3]Long F, Peng H, Liu X, Kim SK, Myers E: A 3D digital atlas of C. elegans and its application to single-cell analyses. Nat Methods 2009, 6(9):667-672.
• [4]Kristensen E, Parsons TE, Hallgramsson B, Boyd SK: A novel 3-D image-based morphological method for phenotypic analysis. Biomedical Engineering, IEEE Transactions 2008, 55(12):2826-2831.
• [5]Peng H, Chung P, Long F, Qu L, Jenett A, Seeds AM, Myers EW, Simpson JH: BrainAligner: 3D registration atlases of Drosophila brains. Nat Methods 2011, 8(6):493-500.
• [6]Shen D, Davatzikos C: HAMMER: hierarchical attribute matching mechanism for elastic registration. IEEE Trans Med Imaging 2002, 21(11):1421-1439.
• [7]Li B, Christensen GE, Hoffman EA, McLennan G, Reinhardt JM: Establishing a normative atlas of the human lung: intersubject warping and registration of volumetric CT images. Acad Radiol 2003, 10(3):255-265.
• [8]Hammond P: The use of 3D face shape modelling in dysmorphology. Arch Dis Child 2007, 92(12):1120.
• [9]Hammond P, Hutton T, Allanson J, Buxton B, Karmiloff-Smith A, Patton M, Pober B, Smith A, Tassabehji M: 3D dense surface models identify the most discriminating facial features in dysmorphic syndromes. Toronto, Canada: 54th Annual Meeting of the American Society for Human Genetics; 2004.
• [10]Hammond P, Hutton TJ, Allanson JE, Buxton B, Campbell LE, Clayton-Smith J, Donnai D, Karmiloff-Smith A, Metcalfe K, Murphy KC, et al.: Discriminating power of localized three-dimensional facial morphology. Am J Hum Genet 2005, 77(6):999-1010.
• [11]Hammond P, Hutton TJ, Allanson JE, Campbell LE, Hennekam RC, Holden S, Patton MA, Shaw A, Temple IK, Trotter M, et al.: 3D analysis of facial morphology. Am J Med Genet A 2004, 126A(4):339-348.
• [12]Hutton TJ, Buxton BF, Hammond P: Automated registration of 3D faces using dense surface models. Citeseer: Proceedings of British Machine Vision Conference; 2003:439-448.
• [13]Hutton TJ, Buxton BF, Hammond P, Potts HWW: Estimating average growth trajectories in shape-space using kernel smoothing. Medical Imaging, IEEE Transactions on 2003 2003, 22(6):747-753.
• [14]Klein A, Andersson J, Ardekani BA, Ashburner J, Avants B, Chiang MC, Christensen GE, Collins DL, Gee J, Hellier P, et al.: Evaluation of 14 nonlinear deformation algorithms applied to human brain MRI registration. Neuro Image 2009, 46(3):786-802.
• [15]Farkas LG, Katic MJ, Forrest CR: International anthropometric study of facial morphology in various ethnic groups/races. J Craniofac Surg 2005, 16(4):615.
• [16]Weinberg SM, Neiswanger K, Richtsmeier JT, Maher BS, Mooney MP, Siegel MI, Marazita ML: Three‒dimensional morphometric analysis of craniofacial shape in the unaffected relatives of individuals with nonsyndromic orofacial clefts: a possible marker for genetic susceptibility. Am J Med Genet A 2008, 146(4):409-420.
• [17]Hammond P, Forster-Gibson C, Chudley AE, Allanson JE, Hutton TJ, Farrell SA, McKenzie J, Holden JJA, Lewis MES: Face and brain asymmetry in autism spectrum disorders. Mol Psychiatry 2008, 13(6):614-623.
• [18]Albert AM, Ricanek K Jr, Patterson E: A review of the literature on the aging adult skull and face: implications for forensic science research and applications. Forensic Sci Int 2007, 172(1):1-9.
• [19]Kayser M, De Knijff P: Improving human forensics through advances in genetics, genomics and molecular biology. Nat Rev Genet 2011, 12(3):179-192.
• [20]Meyer-Marcotty P, Alpers GW, Gerdes A, Stellzig-Eisenhauer A: Impact of facial asymmetry in visual perception: a 3-dimensional data analysis. Am J Orthod Dentofacial Orthop 2010, 137(2):168-e161.
• [21]Little AC, Jones BC, DeBruine LM: Facial attractiveness: evolutionary based research. Philosophical Transactions of the Royal Society B: Biological Sciences 2011, 366(1571):1638-1659.
• [22]Ramanathan N, Chellappa R, Biswas S: Computational methods for modeling facial aging: a survey. J Vis Lang Comput 2009, 20(3):131-144.
• [23]Fu Y, Guo G, Huang TS: Age synthesis and estimation via faces: a survey. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2010 2010, 32(11):1955-1976.
• [24]Liu F, van der Lijn F, Schurmann C, Zhu G, Chakravarty MM, Hysi PG, Wollstein A, Lao O, de Bruijne M, Ikram MA, et al.: A genome-wide association study identifies five loci influencing facial morphology in Europeans. PLoS Genet 2012, 8(9):e1002932.
• [25]Paternoster L, Zhurov AI, Toma AM, Kemp JP, St Pourcain B, Timpson NJ, McMahon G, McArdle W, Ring SM, Smith GD, et al.: Genome-wide association study of three-dimensional facial morphology identifies a variant in PAX3 associated with nasion position. Am J Hum Genet 2012, 90(3):478-485.
• [26]Richardson ER: Racial differences in dimensional traits of the human face. Angle Orthod 1980, 50(4):301-311.
• [27]Allanson JE, O’Hara P, Farkas LG, Nair RC: Anthropometric craniofacial pattern profiles in down syndrome. Am J Med Genet 1993, 47(5):748-752.
• [28]Dryden I, Mardia K: Statistical analysis of shape. Chichester: Wiley; 1998.
• [29]Yamaguchi T, Maki K, Shibasaki Y: Growth hormone receptor gene variant and mandibular height in the normal Japanese population. Am J Orthod Dentofacial Orthop 2001, 119(6):650-653.
• [30]Tomoyasu Y, Yamaguchi T, Tajima A, Nakajima T, Inoue I, Maki K: Further evidence for an association between mandibular height and the growth hormone receptor gene in a Japanese population. Am J Orthod Dentofacial Orthop 2009, 136(4):536-541.
• [31]Weinberg S, Naidoo S, Bardi K, Brandon C, Neiswanger K, Resick J, Martin R, Marazita M: Face shape of unaffected parents with cleft affected offspring: combining three‒dimensional surface imaging and geometric morphometrics. Orthod Craniofac Res 2009, 12(4):271-281.
• [32]Ermakov S, Rosenbaum MG, Malkin I, Livshits G: Family-based study of association between ENPP1 genetic variants and craniofacial morphology. Ann Hum Biol 2010, 37(6):754-766.
• [33]Boehringer S, Van Der Lijn F, Liu F, Günther M, Sinigerova S, Nowak S, Ludwig KU, Herberz R, Klein S, Hofman A: Genetic determination of human facial morphology: links between cleft-lips and normal variation. Eur J Hum Genet 2011, 19(11):1192-1197.
• [34]Besl PJ, McKay ND: A method for registration of 3-D shapes. IEEE Trans Pattern Anal Mach Intell 1992, 14(2):239-256.
• [35]Creusot C, Pears N, Austin J: 3D face landmark labelling. New York, NY, USA: ACM; 2010:27-32. [Proceedings of the ACM workshop on 3D object retrieval]
• [36]Dibeklioglu H, Salah AA, Akarun L: 3D facial landmarking under expression, pose, and occlusion variations. Arlington, VA, USA: IEEE; 2008:1-6. [2nd IEEE International Conference on Biometrics: Theory, Applications and Systems: 2008]
• [37]Szeptycki P, Ardabilian M, Chen L: A coarse-to-fine curvature analysis-based rotation invariant 3D face landmarking. Washington, DC, USA: IEEE; 2009:1-6. [IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems: 2009]
• [38]Chang KI, Bowyer W, Flynn PJ: Multiple nose region matching for 3D face recognition under varying facial expression. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2006 2006, 28(10):1695-1700.
• [39]Salah AA, Inar H, Akarun L, Sankur B: Robust facial landmarking for registration. Ann Telecommun 2007, 62(1–2):1608-1633.
• [40]Irfanoglu MO, Gokberk B, Akarun L: 3D shape-based face recognition using automatically registered facial surfaces. Washington, DC, USA: IEEE; 2004:183-186. [Proceedings of the 17th International Conference on Pattern Recognition: 2004]
• [41]Schneider DC, Eisert P, Herder J, Magnor M, Grau O: Algorithms for automatic and robust registration of 3d head scans. Journal of Virtual Reality and Broadcasting 2010, 7:7.
• [42]Colombo A, Cusano C, Schettini R: 3D face detection using curvature analysis. Pattern recognition 2006, 39(3):444-455.
• [43]Kakadiaris I, Passalis G, Toderici G, Murtuza N, Theoharis T: 3D face recognition. Edinburgh, UK: Proceedings of the British Machine Vision Conference: 2006; 2006:869-868.
• [44]Nair P, Cavallaro A: Matching 3D faces with partial data. Leeds, UK: Proc British Machine Vision Conference: 2008; 2008:1-4.
• [45]Abate AF, Nappi M, Riccio D, Sabatino G: 2D And 3D face recognition: a survey. Pattern Recognit Lett 2007, 28(14):1885-1906.
• [46]Wang Y, Chua CS, Ho YK: Facial feature detection and face recognition from 2D and 3D images. Pattern Recognit Lett 2002, 23(10):1191-1202.
• [47]D’Hose J, Colineau J, Bichon C, Dorizzi B: Precise localization of landmarks on 3d faces using gabor wavelets. Crystal City, VA, USA: IEEE; 2007:1-6. [First IEEE International Conference on Biometrics: Theory, Applications, and Systems]
• [48]Bookstein FL: Principal warps: thin-plate splines and the decomposition of deformations. Pattern Analysis and Machine Intelligence, IEEE Transactions on 1989 1989, 11(6):567-585. Washington, DC, USA
• [49]Sun Y, Yin L: Automatic pose estimation of 3D facial models. Tampa, FL, USA: IEEE; 2008:1-4. [19th International Conference on Pattern Recognition: 2008]
• [50]Lu X, Jain AK, Colbry D: Matching 2.5 D face scans to 3D models. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2006 2006, 28(1):31-43. Washington, DC, USA
• [51]Salah AA, Alyz N, Akarun L: Registration of three-dimensional face scans with average face models. Journal of Electronic Imaging 2008, 17:011006.
• [52]ter Haar FB, Veltkamp RC: A 3D face matching framework for facial curves. Graph Model 2009, 71(2):77-91.
• [53]Mian AS, Bennamoun M, Owens R: An efficient multimodal 2D-3D hybrid approach to automatic face recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on 2007, 29(11):1927-1943.
• [54]Tsalakanidou F, Tzovaras D, Strintzis MG: Use of depth and colour eigenfaces for face recognition. Pattern Recognit Lett 2003, 24(9–10):1427-1435.
• [55]Mian AS, Bennamoun M, Owens R: Keypoint detection and local feature matching for textured 3D face recognition. Int J Comput Vis 2008, 79(1):1-12.
• [56]Chang K, Bowyer K, Flynn P: Face recognition using 2D and 3D facial data. Santa Barbara, California, USA: ACM Workshop on Multimodal User Authentication: 2003; 2003:25-32.
• [57]Turk MA, Pentland AP: Face recognition using eigenfaces. Maui, HI, USA: IEEE; 1991:586-591. [IEEE Computer Society Conference on Computer Vision and Pattern Recognition: 1991]
• [58]Alliez P, Ucelli G, Gotsman C, Attene M: Recent advances in remeshing of surfaces. Shape analysis and structuring 2008, 53-82.
• [59]Hutton TJ, Buxton BR, Hammond P: Dense surface point distribution models of the human face. Kauai, HI, USA: IEEE; 2001:153-160. [IEEE Workshop on Mathematical Methods in Biomedical Image Analysis: 2001]
• [60]Johnson H, Christensen G: Landmark and intensity-based, consistent thin-plate spline image registration. London, UK: Springer; 2001:329-343. [Information Processing in Medical Imaging: 2001]
• [61]Schneider PJ, Eberly DH: Geometric tools for computer graphics. Morgan Kaufmann Pub; 2003.
• [62]Gower JC: Generalized procrustes analysis. Psychometrika 1975, 40(1):33-51.
• [63]Perakis P, Passalis G, Theoharis T, Kakadiaris IA: 3D Facial landmark detection & face registration. Tech Rep: University of Athens; 2011.
• [64]Xu S, Huang W, Qian J, Jin L: Analysis of genomic admixture in Uyghur and its implication in mapping strategy. Am J Hum Genet 2008, 82(4):883-894.
• [65]Wu G, Yap PT, Kim M, Shen D: TPS-HAMMER: improving HAMMER registration algorithm by soft correspondence matching and thin-plate splines based deformation interpolation. NeuroImage 2010, 49(3):2225-2233.
• [66]Lao Z, Shen D, Xue Z, Karacali B, Resnick SM, Davatzikos C: Morphological classification of brains via high-dimensional shape transformations and machine learning methods. NeuroImage 2004, 21(1):46-57.
• [67]Yang JS, Awasaki T, Yu HH, He Y, Ding P, Kao JC, Lee T: Diverse neuronal lineages make stereotyped contributions to the Drosophila locomotor control center, the central complex. J Comp Neurol 2013, 521(12):Spc1.
• [68]Helmstaedter M, Mitra PP: Computational methods and challenges for large-scale circuit mapping. Curr Opin Neurobiol 2012, 22(1):162-169.