Skip to main content
Top
Published in: Soft Computing 3/2011

01-03-2011 | Focus

2D face fitting-assisted 3D face reconstruction for pose-robust face recognition

Authors: Liting Wang, Liu Ding, Xiaoqing Ding, Chi Fang

Published in: Soft Computing | Issue 3/2011

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Recent face recognition algorithm can achieve high accuracy when the tested face samples are frontal. However, when the face pose changes largely, the performance of existing methods drop drastically. Efforts on pose-robust face recognition are highly desirable, especially when each face class has only one frontal training sample. In this study, we propose a 2D face fitting-assisted 3D face reconstruction algorithm that aims at recognizing faces of different poses when each face class has only one frontal training sample. For each frontal training sample, a 3D face is reconstructed by optimizing the parameters of 3D morphable model (3DMM). By rotating the reconstructed 3D face to different views, pose virtual face images are generated to enlarge the training set of face recognition. Different from the conventional 3D face reconstruction methods, the proposed algorithm utilizes automatic 2D face fitting to assist 3D face reconstruction. We automatically locate 88 sparse points of the frontal face by 2D face-fitting algorithm. Such 2D face-fitting algorithm is so-called Random Forest Embedded Active Shape Model, which embeds random forest learning into the framework of Active Shape Model. Results of 2D face fitting are added to the 3D face reconstruction objective function as shape constraints. The optimization objective energy function takes not only image intensity, but also 2D fitting results into account. Shape and texture parameters of 3DMM are thus estimated by fitting the 3DMM to the 2D frontal face sample, which is a non-linear optimization problem. We experiment the proposed method on the publicly available CMUPIE database, which includes faces viewed from 11 different poses, and the results show that the proposed method is effective and the face recognition results toward pose variants are promising.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Bellhumer PN, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class-specific linear projection. IEEE Trans Pattern Anal Mach Intell Spec Issue Face Recognit 17(7):711–720CrossRef Bellhumer PN, Hespanha J, Kriegman D (1997) Eigenfaces vs. fisherfaces: recognition using class-specific linear projection. IEEE Trans Pattern Anal Mach Intell Spec Issue Face Recognit 17(7):711–720CrossRef
go back to reference Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH, pp 187–194 Blanz V, Vetter T (1999) A morphable model for the synthesis of 3D faces. In: Proceedings of SIGGRAPH, pp 187–194
go back to reference Blanz V, Vetter T (2003) Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074CrossRef Blanz V, Vetter T (2003) Face recognition based on fitting a 3D morphable model. IEEE Trans Pattern Anal Mach Intell 25(9):1063–1074CrossRef
go back to reference Blanz V, Grother P, Phillips PJ, Vetter T (2005) Face recognition based on frontal views generated from nonfrontal images. IEEE Conf Comput Vis Pattern Recognit 2:454–461 Blanz V, Grother P, Phillips PJ, Vetter T (2005) Face recognition based on frontal views generated from nonfrontal images. IEEE Conf Comput Vis Pattern Recognit 2:454–461
go back to reference Breuer P, Kim KI, Kienzle W, Scholkopf B, Blanz V (2008) Automatic 3D face reconstruction from single images or video. In: Automatic face and gesture recognition (FG) IEEE international conference, pp 1–8 Breuer P, Kim KI, Kienzle W, Scholkopf B, Blanz V (2008) Automatic 3D face reconstruction from single images or video. In: Automatic face and gesture recognition (FG) IEEE international conference, pp 1–8
go back to reference Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052CrossRef Brunelli R, Poggio T (1993) Face recognition: features versus templates. IEEE Trans Pattern Anal Mach Intell 15(10):1042–1052CrossRef
go back to reference Cappelli R, Maio D, Maltoni D (2002) Subspace classification for face recognition. In: Proceeding workshop on biometric authentication, pp 133–141 Cappelli R, Maio D, Maltoni D (2002) Subspace classification for face recognition. In: Proceeding workshop on biometric authentication, pp 133–141
go back to reference Chai XJ, Shan SG, Qing LY, Chen XL, Gao W (2006) Pose and illumination invariant face recognition based on 3D face reconstruction. J Softw 17(3):525–534MATHCrossRef Chai XJ, Shan SG, Qing LY, Chen XL, Gao W (2006) Pose and illumination invariant face recognition based on 3D face reconstruction. J Softw 17(3):525–534MATHCrossRef
go back to reference Cootes TF, Taylor C, Cooper D, Graham J (1995) Active shape models: their training and their applications. Comput Vis Image Underst 61(1):38–59CrossRef Cootes TF, Taylor C, Cooper D, Graham J (1995) Active shape models: their training and their applications. Comput Vis Image Underst 61(1):38–59CrossRef
go back to reference Cootes TF, Edwards G, Taylor C (2001) Active Appearance Models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685CrossRef Cootes TF, Edwards G, Taylor C (2001) Active Appearance Models. IEEE Trans Pattern Anal Mach Intell 23(6):681–685CrossRef
go back to reference Franco A, Maio D, Maltoni D (2008) 2D face recognition based on supervised subspace learning from 3D models. Pattern Recognit 41:3822–3833MATHCrossRef Franco A, Maio D, Maltoni D (2008) 2D face recognition based on supervised subspace learning from 3D models. Pattern Recognit 41:3822–3833MATHCrossRef
go back to reference Gonzalez-Jimenez D, Alba-Castro JL (2007) Face recognition through point distribution models and facial symmetry. IEEE Trans Inf Forensics Secur 2(3):413–429CrossRefMathSciNet Gonzalez-Jimenez D, Alba-Castro JL (2007) Face recognition through point distribution models and facial symmetry. IEEE Trans Inf Forensics Secur 2(3):413–429CrossRefMathSciNet
go back to reference Guo GD, Li SZ, Chan KL (2000) Face recognition by support vector machines. In: Proceedings of automatic face and gesture recognition, pp 196–201 Guo GD, Li SZ, Chan KL (2000) Face recognition by support vector machines. In: Proceedings of automatic face and gesture recognition, pp 196–201
go back to reference Gurbuz S, Inoue N (2007) Real-time head pose estimation using reconstructed 3d face data from stereo image pair. Acoustics Speech and Signal Processing (ICASSP), IEEE international conference, vol 2, pp 785–788 Gurbuz S, Inoue N (2007) Real-time head pose estimation using reconstructed 3d face data from stereo image pair. Acoustics Speech and Signal Processing (ICASSP), IEEE international conference, vol 2, pp 785–788
go back to reference Hu YX, Jiang DL, Yan SC, Zhang L, Zhang HJ (2004) Automatic 3D reconstruction for face recognition. In: Proceedings of IEEE automatic face and gesture recognition, pp 843–848 Hu YX, Jiang DL, Yan SC, Zhang L, Zhang HJ (2004) Automatic 3D reconstruction for face recognition. In: Proceedings of IEEE automatic face and gesture recognition, pp 843–848
go back to reference Huang C, Ai HZ, Li Y, Lao SH (2007) High performance rotation invariant multiview face detection. IEEE Trans Pattern Anal Mach Intell 29(4):671–686CrossRef Huang C, Ai HZ, Li Y, Lao SH (2007) High performance rotation invariant multiview face detection. IEEE Trans Pattern Anal Mach Intell 29(4):671–686CrossRef
go back to reference Lepetit V, Fua P (2006) Keypoint recognition using randomized trees. IEEE Trans Pattern Anal Mach Intell, pp 1465–1479 Lepetit V, Fua P (2006) Keypoint recognition using randomized trees. IEEE Trans Pattern Anal Mach Intell, pp 1465–1479
go back to reference Ma Y, Ding XQ, Wang ZE, Wang N (2004) Robust precise eye location under probabilistic framework. IEEE international conference on automatic face and gesture recognition, pp 339–344 Ma Y, Ding XQ, Wang ZE, Wang N (2004) Robust precise eye location under probabilistic framework. IEEE international conference on automatic face and gesture recognition, pp 339–344
go back to reference Michael J, Jones T (1998) Multidimensional morphable models: a framework for representing and matching object classes. Int J Comput Vis 2(29):107–131 Michael J, Jones T (1998) Multidimensional morphable models: a framework for representing and matching object classes. Int J Comput Vis 2(29):107–131
go back to reference Multimedia, Intelligent Software Technology Beijing Municipal Key Laboratory BJUT (2005) The BJUT-3D large-scale chinese face database. Technical report Multimedia, Intelligent Software Technology Beijing Municipal Key Laboratory BJUT (2005) The BJUT-3D large-scale chinese face database. Technical report
go back to reference Phong BT (1975) Illumination for computer-generated images. Commun ACM 18(6):311–317CrossRef Phong BT (1975) Illumination for computer-generated images. Commun ACM 18(6):311–317CrossRef
go back to reference Suen CY, Langaroudi AZ, Feng CH, Mao YX (2007) A survey of techniques for face reconstruction. Systems, Man and Cybernetics (ISIC), IEEE international conference, pp 3554–3560 Suen CY, Langaroudi AZ, Feng CH, Mao YX (2007) A survey of techniques for face reconstruction. Systems, Man and Cybernetics (ISIC), IEEE international conference, pp 3554–3560
go back to reference Sung WP, Heo JG, Savvides M (2008) 3D Face econstruction from a single 2D face image. Computer vision and pattern recognition workshops (CVPR workshops), IEEE computer society conference, pp 1–8 Sung WP, Heo JG, Savvides M (2008) 3D Face econstruction from a single 2D face image. Computer vision and pattern recognition workshops (CVPR workshops), IEEE computer society conference, pp 1–8
go back to reference Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc Comput Vis Pattern Recognit 1:511–518 Viola P, Jones M (2001) Rapid object detection using a boosted cascade of simple features. Proc Comput Vis Pattern Recognit 1:511–518
go back to reference Wang ZE, Ding XQ, Fang C (2008) Pose-adaptive lda-based face recognition. In: Proceedings of international conference on pattern recognition Wang ZE, Ding XQ, Fang C (2008) Pose-adaptive lda-based face recognition. In: Proceedings of international conference on pattern recognition
go back to reference Wiskott L, Fellous JM, Malsburg VD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779CrossRef Wiskott L, Fellous JM, Malsburg VD (1997) Face recognition by elastic bunch graph matching. IEEE Trans Pattern Anal Mach Intell 19:775–779CrossRef
go back to reference Zhao M, Chua TS, Sim T (2006) Morphable face reconstruction with multiple images. Automatic face and gesture recognition (FGR), International conference, pp 597–602 Zhao M, Chua TS, Sim T (2006) Morphable face reconstruction with multiple images. Automatic face and gesture recognition (FGR), International conference, pp 597–602
go back to reference Zheng Y, Chang JL, Zheng ZG, Wang ZF (2007) 3D face reconstruction from stereo: a model-based approach. Image processing (ICIP), IEEE international conference, vol 3, pp 65–68 Zheng Y, Chang JL, Zheng ZG, Wang ZF (2007) 3D face reconstruction from stereo: a model-based approach. Image processing (ICIP), IEEE international conference, vol 3, pp 65–68
Metadata
Title
2D face fitting-assisted 3D face reconstruction for pose-robust face recognition
Authors
Liting Wang
Liu Ding
Xiaoqing Ding
Chi Fang
Publication date
01-03-2011
Publisher
Springer-Verlag
Published in
Soft Computing / Issue 3/2011
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-009-0523-0

Other articles of this Issue 3/2011

Soft Computing 3/2011 Go to the issue

Premium Partner