Skip to main content
SpringerLink
Log in

Principal Curvature Measures Estimation and Application to 3D Face Recognition

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

This paper presents an effective 3D face keypoint detection, description and matching framework based on three principle curvature measures. These measures give a unified definition of principle curvatures for both smooth and discrete surfaces. They can be reasonably computed based on the normal cycle theory and the geometric measure theory. The strong theoretical basis of these measures provides us a solid discrete estimation method on real 3D face scans represented as triangle meshes. Based on these estimated measures, the proposed method can automatically detect a set of sparse and discriminating 3D facial feature points. The local facial shape around each 3D feature point is comprehensively described by histograms of these principal curvature measures. To guarantee the pose invariance of these descriptors, three principle curvature vectors of these principle curvature measures are employed to assign the canonical directions. Similarity comparison between faces is accomplished by matching all these curvature-based local shape descriptors using the sparse representation-based reconstruction method. The proposed method was evaluated on three public databases, i.e. FRGC v2.0, Bosphorus, and Gavab. Experimental results demonstrated that the three principle curvature measures contain strong complementarity for 3D facial shape description, and their fusion can largely improve the recognition performance. Our approach achieves rank-one recognition rates of 99.6, 95.7, and 97.9% on the neutral subset, expression subset, and the whole FRGC v2.0 databases, respectively. This indicates that our method is robust to moderate facial expression variations. Moreover, it also achieves very competitive performance on the pose subset (over 98.6% except Yaw 90°) and the occlusion subset (98.4%) of the Bosphorus database. Even in the case of extreme pose variations like profiles, it also significantly outperforms the state-of-the-art approaches with a recognition rate of 57.1%. The experiments carried out on the Gavab databases further demonstrate the robustness of our method to varies head pose variations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11

Similar content being viewed by others

References

  1. Alyuz, N., Gokberk, B., Akarun, L.: Regional registration for expression resistant 3-d face recognition. IEEE Trans. Inf. Forensics Secur. 5(3), 425–440 (2010)

    Article  Google Scholar 

  2. Alyüz, N., Gökberk, B., Dibeklioğlu, H., Savran, A., Salah, A.A., Akarun, L., Sankur, B.: 3D face recognition benchmarks on the bosphorus database with focus on facial expressions. In: European Workshop on Biometrics and Identity Management, pp. 57–66. Springer (2008)

  3. Amberg, B., Knothe, R., Vetter, T.: Expression invariant 3d face recognition with a morphable model. In: IEEE International Conference on Automatic Face and Gesture Recognition, pp. 1–6. IEEE (2008)

  4. Ballihi, L., Ben Amor, B., Daoudi, M., Srivastava, A., Aboutajdine, D.: Boosting 3-d-geometric features for efficient face recognition and gender classification. IEEE Trans. Inf. Forensics Secur. 7(6), 1766–1779 (2012)

    Article  Google Scholar 

  5. Berretti, S., Del Bimbo, A., Pala, P.: 3D face recognition using isogeodesic stripes. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2162–2177 (2010)

    Article  Google Scholar 

  6. Berretti, S., Werghi, N., Del Bimbo, A., Pala, P.: Matching 3d face scans using interest points and local histogram descriptors. Comput. Graph. 37(5), 509–525 (2013)

    Article  Google Scholar 

  7. Borrelli, V., Cazals, F., Morvan, J.M.: On the angular defect of triangulations and the pointwise approximation of curvatures. Comput. Aided Geom. Des. 20(6), 319–341 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  8. Bowyer, K.W., Chang, K., Flynn, P.: A survey of approaches and challenges in 3d and multi-modal 3d+ 2d face recognition. Comput. Vis. Image Underst. 101(1), 1–15 (2006)

    Article  Google Scholar 

  9. Bronstein, A.M., Bronstein, M.M., Guibas, L.J., Ovsjanikov, M.: Shape google: geometric words and expressions for invariant shape retrieval. ACM Trans. Graph. (TOG) 30(1), 1 (2011)

    Article  Google Scholar 

  10. Bronstein, A.M., Bronstein, M.M., Kimmel, R.: Expression-invariant 3d face recognition. In: International Conference on Audio-and Video-Based Biometric Person Authentication, pp. 62–70. Springer (2003)

  11. Bronstein, A.M., Bronstein, M.M., Spira, A., Kimmel, R.: Face recognition from facial surface metric. In: European Conference on Computer Vision, pp. 225–237. Springer (2004)

  12. Bronstein, M.M., Kokkinos, I.: Scale-invariant heat kernel signatures for non-rigid shape recognition. In: 2010 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1704–1711. IEEE (2010)

  13. Chang, K.I., Bowyer, K.W., Flynn, P.J.: Adaptive rigid multi-region selection for handling expression variation in 3d face recognition. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 157–157 (2005)

  14. Chang, K.I., Bowyer, K.W., Flynn, P.J.: Multiple nose region matching for 3d face recognition under varying facial expression. IEEE Trans. Pattern Anal. Mach. Intell. 28(10), 1695–1700 (2006)

    Article  Google Scholar 

  15. Chen, B.: Geometry of Submanifolds, vol. 22. M. Dekker (1973)

  16. Chua, C.S., Jarvis, R.: Point signatures: a new representation for 3d object recognition. Int. J. Comput. Vis. 25(1), 63–85 (1997)

    Article  Google Scholar 

  17. Cohen-Steiner, D., Morvan, J.M.: Restricted delaunay triangulations and normal cycle. In: 9th Annual Symposium on Computational Geometry, pp. 312–321. ACM (2003)

  18. Cohen-Steiner, D., Morvan, J.M., et al.: Second fundamental measure of geometric sets and local approximation of curvatures. J. Differ. Geom. 74(3), 363–394 (2006)

    Article  MathSciNet  MATH  Google Scholar 

  19. Colombo, A., Cusano, C., Schettini, R.: Three-dimensional occlusion detection and restoration of partially occluded faces. J. Math. Imaging Vis. 40(1), 105–119 (2011)

    Article  MathSciNet  MATH  Google Scholar 

  20. Conde, C., Rodríguez-Aragón, L.J., Cabello, E.: Automatic 3d face feature points extraction with spin images. In: International Conference Image Analysis and Recognition, pp. 317–328. Springer (2006)

  21. Dibeklioğlu, H., Gökberk, B., Akarun, L.: Nasal region-based 3d face recognition under pose and expression variations. In: International Conference on Biometrics, pp. 309–318. Springer (2009)

  22. do Carmo, M.P.: Riemannian Geometry. Birkhauser, Boston (1992)

  23. Drira, H., Amor, B.B., Daoudi, M., Srivastava, A.: Pose and expression-invariant 3d face recognition using elastic radial curves. In: British Machine Vision Conference, pp. 1–11 (2010)

  24. Drira, H., Amor, B.B., Srivastava, A., Daoudi, M., Slama, R.: 3D face recognition under expressions, occlusions, and pose variations. IEEE Trans. Pattern Anal. Mach. Intell. 35(9), 2270–2283 (2013)

    Article  Google Scholar 

  25. Faltemier, T.C., Bowyer, K.W., Flynn, P.J.: A region ensemble for 3-d face recognition. IEEE Trans. Inf. Forensics Secur. 3(1), 62–73 (2008)

    Article  Google Scholar 

  26. Fu, J.H.: Monge-ampere functions. Research rep./Centre for math. analysis; CMA-R16-88 (1988)

  27. Goldfeather, J., Interrante, V.: A novel cubic-order algorithm for approximating principal direction vectors. ACM Trans. Graph. 23(1), 45–63 (2004)

    Article  Google Scholar 

  28. Gordon, G.G.: Face recognition based on depth and curvature features. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition., pp. 808–810 (1992)

  29. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J.: 3D object recognition in cluttered scenes with local surface features: a survey. IEEE Trans. Pattern Anal. Mach. Intell. 36(11), 2270–2287 (2014)

    Article  Google Scholar 

  30. Guo, Y., Bennamoun, M., Sohel, F., Lu, M., Wan, J., Kwok, N.M.: A comprehensive performance evaluation of 3d local feature descriptors. Int. J. Comput. Vis. 116(1), 66–89 (2016)

    Article  MathSciNet  Google Scholar 

  31. Hajati, F., Raie, A.A., Gao, Y.: 2.5D face recognition using patch geodesic moments. Pattern Recogn. 45(3), 969–982 (2012)

    Article  MATH  Google Scholar 

  32. Hamann, B.: Curvature approximation for triangulated surfaces. In: Geometric Modelling, pp. 139–153. Springer (1993)

  33. Huang, D., Ardabilian, M., Wang, Y., Chen, L.: A novel geometric facial representation based on multi-scale extended local binary patterns. In: IEEE International Conference on Automatic Face and Gesture Recognition and Workshops, pp. 1–7 (2011)

  34. Huang, D., Ardabilian, M., Wang, Y., Chen, L.: 3-D face recognition using eLBP-based facial description and local feature hybrid matching. IEEE Trans. Inf. Forensics Secur. 7(5), 1551–1565 (2012)

    Article  Google Scholar 

  35. Huang, D., Zhang, G., Ardabilian, M., Wang, Y., Chen, L.: 3D face recognition using distinctiveness enhanced facial representations and local feature hybrid matching. In: 2010 Fourth IEEE International Conference on Biometrics: Theory Applications and Systems (BTAS), pp. 1–7. IEEE (2010)

  36. Hubel, D.H., Wiesel, T.N.: Receptive fields, binocular interaction and functional architecture in the cat’s visual cortex. J. Physiol. 160(1), 106–154 (1962)

    Article  Google Scholar 

  37. Kakadiaris, I.A., Passalis, G., Toderici, G., Murtuza, M.N., Lu, Y., Karampatziakis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: an annotated deformable model approach. IEEE Trans. Pattern Anal. Mach. Intell. 29(4), 640–649 (2007)

    Article  Google Scholar 

  38. Li, H.: Towards three-dimensional face recognition in the real. Ph.D. thesis, Citeseer (2014)

  39. Li, H., Huang, D., Morvan, J.M., Chen, L., Wang, Y.: Expression-robust 3d face recognition via weighted sparse representation of multi-scale and multi-component local normal patterns. Neurocomputing 133, 179–193 (2014)

    Article  Google Scholar 

  40. Li, H., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: Towards 3d face recognition in the real: a registration-free approach using fine-grained matching of 3d keypoint descriptors. Int. J. Comput. Vis. 113(2), 128–142 (2015)

    Article  MathSciNet  Google Scholar 

  41. Li, H., Zeng, W., Morvan, J.M., Chen, L., Gu, X.D.: Surface meshing with curvature convergence. IEEE Trans. Vis. Comput. Graph. 20(6), 919–934 (2014)

    Article  Google Scholar 

  42. Li, X., Jia, T., Zhang, H.: Expression-insensitive 3d face recognition using sparse representation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2575–2582 (2009)

  43. Li, X., Zhang, H.: Adapting geometric attributes for expression-invariant 3d face recognition. In: IEEE International Conference on Shape Modeling and Applications, pp. 21–32 (2007)

  44. Lo, T.W.R., Siebert, J.P.: Local feature extraction and matching on range images: 2.5D sift. Comput. Vis. Image Underst. 113(12), 1235–1250 (2009)

    Article  Google Scholar 

  45. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)

    Article  Google Scholar 

  46. Lu, X., Jain, A.: Deformation modeling for robust 3d face matching. IEEE Trans. Pattern Anal. Mach. Intell. 30(8), 1346–1357 (2008)

    Article  Google Scholar 

  47. Lu, X., Jain, A.K., Colbry, D.: Matching 2.5d face scans to 3d models. IEEE Trans. Pattern Anal. Mach. Intell. 28(1), 31–43 (2006)

    Article  Google Scholar 

  48. Maes, C., Fabry, T., Keustermans, J., Smeets, D., Suetens, P., Vandermeulen, D.: Feature detection on 3d face surfaces for pose normalisation and recognition. In: Fourth IEEE International Conference on Biometrics: Theory Applications and Systems, pp. 1–6 (2010)

  49. Mahoor, M.H., Abdel-Mottaleb, M.: Face recognition based on 3d ridge images obtained from range data. Pattern Recogn. 42(3), 445–451 (2009)

    Article  MATH  Google Scholar 

  50. Meyer, M., Desbrun, M., Schröder, P., Barr, A.H.: Discrete differential-geometry operators for triangulated 2-manifolds. In: Visualization and Mathematics III, pp. 35–57. Springer (2003)

  51. Mian, A., Bennamoun, M., Owens, R.: An efficient multimodal 2d–3d hybrid approach to automatic face recognition. IEEE Trans. Pattern Anal. Mach. Intell. 29(11), 1927–1943 (2007)

    Article  Google Scholar 

  52. Mian, A.S., Bennamoun, M., Owens, R.: Keypoint detection and local feature matching for textured 3d face recognition. Int. J. Comput. Vis. 79(1), 1–12 (2008)

    Article  Google Scholar 

  53. Moreno, A.B., Sánchez, A.: Gavabdb: a 3d face database. In: Workshop on Biometrics on the Internet, pp. 75–80 (2004)

  54. Morvan, J.M.: Generalized Curvatures. Springer, Berlin (2008)

    Book  MATH  Google Scholar 

  55. Morvan, J.M., Thibert, B.: Approximation of the normal vector field and the area of a smooth surface. Discret. Comput. Geom. 32(3), 383–400 (2004)

    Article  MathSciNet  MATH  Google Scholar 

  56. Mpiperis, I., Malassiotis, S., Strintzis, M.G.: Bilinear models for 3-d face and facial expression recognition. IEEE Trans. Inf. Forensics Secur. 3(3), 498–511 (2008)

    Article  Google Scholar 

  57. Pears, N., Liu, Y., Bunting, P.: 3D imaging, analysis and applications, vol. 3. Springer, Berlin (2012)

    Book  Google Scholar 

  58. Peter, W.: Normal cycle and integral curvature for polyhedra in riemannian manifolds. Differential Geometry, Colloq. Math. Soc. Janos Bolyai (1982)

  59. Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J., Worek, W.: Overview of the face recognition grand challenge. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recog. 1, 947–954 (2005)

    Google Scholar 

  60. Raviv, D., Bronstein, A.M., Bronstein, M.M., Waisman, D., Sochen, N., Kimmel, R.: Equi-affine invariant geometry for shape analysis. J. Math. Imaging Vis. 50(1–2), 144–163 (2014)

    Article  MathSciNet  MATH  Google Scholar 

  61. Razdan, A., Bae, M.: Curvature estimation scheme for triangle meshes using biquadratic bézier patches. Comput. Aided Des. 37(14), 1481–1491 (2005)

    Article  MATH  Google Scholar 

  62. Rusinkiewicz, S.: Estimating curvatures and their derivatives on triangle meshes. In: 2nd IEEE International Symposium on 3D Data Processing, Visualization and Transmission, pp. 486–493 (2004)

  63. Samir, C., Srivastava, A., Daoudi, M.: Three-dimensional face recognition using shapes of facial curves. IEEE Trans. Pattern Anal. Mach. Intell. 28(11), 1858–1863 (2006)

    Article  Google Scholar 

  64. Savran, A., Alyüz, N., Dibeklioglu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: 3D face recognition benchmarks on the bosphorus database with focus on facial expressions. In: Workshop on Biometrics and Identity Management (2008)

  65. Sharma, A., Horaud, R., Cech, J., Boyer, E.: Topologically-robust 3d shape matching based on diffusion geometry and seed growing. In: 2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2481–2488. IEEE (2011)

  66. Smeets, D., Claes, P., Hermans, J., Vandermeulen, D., Suetens, P.: A comparative study of 3-d face recognition under expression variations. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 42(5), 710–727 (2012)

    Article  Google Scholar 

  67. Smeets, D., Keustermans, J., Vandermeulen, D., Suetens, P.: Meshsift: local surface features for 3d face recognition under expression variations and partial data. Comput. Vis. Image Underst. 117(2), 158–169 (2013)

    Article  Google Scholar 

  68. Spivak, M.: Calculus on Manifolds, vol. 1. WA Benjamin, New York (1965)

    MATH  Google Scholar 

  69. Spivak, M.: Comprehensive Introduction to Differential Geometry, vol. 4. University of Tokyo Press, Tokyo (1981)

    MATH  Google Scholar 

  70. Spreeuwers, L.: Fast and accurate 3d face recognition. Int. J. Comput. Vis. 93(3), 389–414 (2011)

    Article  MATH  Google Scholar 

  71. Spreeuwers, L.: Breaking the 99% barrier: optimisation of three-dimensional face recognition. IET Biom. 4(3), 169–178 (2015)

    Article  Google Scholar 

  72. Sun, X., Morvan, J.M.: Curvature measures, normal cycles and asymptotic cones. Actes des rencontres du C.I.R.M. 3(1), 3–10 (2013)

    Article  Google Scholar 

  73. Sun, X., Morvan, J.M.: Asymptotic cones of embedded singular spaces. arXiv preprint arXiv:1501.02639 (2015)

  74. Surazhsky, T., Magid, E., Soldea, O., Elber, G., Rivlin, E.: A comparison of gaussian and mean curvatures estimation methods on triangular meshes. IEEE Int. Conf. Robot. Autom. 1, 1021–1026 (2003)

    Google Scholar 

  75. Szeptycki, P., Ardabilian, M., Chen, L.: A coarse-to-fine curvature analysis-based rotation invariant 3d face landmarking. In: 3rd IEEE International Conference on Biometrics: Theory, Applications, and Systems, pp. 1–6 (2009)

  76. Tanaka, H.T., Ikeda, M., Chiaki, H.: Curvature-based face surface recognition using spherical correlation. principal directions for curved object recognition. In: Third IEEE International Conference on Automatic Face and Gesture Recognition, pp. 372–377 (1998)

  77. Tang, H., Yin, B., Sun, Y., Hu, Y.: 3D face recognition using local binary patterns. Sig. Process. 93(8), 2190–2198 (2013)

    Article  Google Scholar 

  78. Tang, Y., Sun, X., Huang, D., Morvan, J.M., Wang, Y., Chen, L.: 3D face recognition with asymptotic cones based principal curvatures. In: IEEE International Conference on Biometrics (ICB), pp. 466–472 (2015)

  79. Taubin, G.: Estimating the tensor of curvature of a surface from a polyhedral approximation. In: Fifth IEEE International Conference on Computer Vision, pp. 902–907 (1995)

  80. Theisel, H., Rossi, C., Zayer, R., Seidel, H.P.: Normal based estimation of the curvature tensor for triangular meshes. In: 12th IEEE Pacific Conference on Computer Graphics and Applications, pp. 288–297 (2004)

  81. Tola, E., Lepetit, V., Fua, P.: Daisy: an efficient dense descriptor applied to wide-baseline stereo. IEEE Trans. Pattern Anal. Mach. Intell. 32(5), 815–830 (2010)

    Article  Google Scholar 

  82. Wang, Y., Liu, J., Tang, X.: Robust 3d face recognition by local shape difference boosting. IEEE Trans. Pattern Anal. Mach. Intell. 32(10), 1858–1870 (2010)

    Article  Google Scholar 

  83. Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)

    Article  Google Scholar 

  84. Wu, Z., Wang, Y., Pan, G.: 3D face recognition using local shape map. IEEE Int. Conf. Image Process. 3, 2003–2006 (2004)

    Google Scholar 

  85. Xu, G., Bajaj, C.L.: Curvature computation of 2-manifolds in R k. J. Comput. Math. 681–688 (2003)

  86. Xu, Z., Xu, G.: Discrete schemes for Gaussian curvature and their convergence. Comput. Math. Appl. 57(7), 1187–1195 (2009)

    Article  MathSciNet  MATH  Google Scholar 

  87. Zhang, L., Razdan, A., Farin, G., Femiani, J., Bae, M., Lockwood, C.: 3D face authentication and recognition based on bilateral symmetry analysis. Vis. Comput. 22(1), 43–55 (2006)

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported in part by the French research agency, l’Agence Nationale de Recherche (ANR), through the Biofence project under the Grant ANR-13-INSE-0004-02. Huibin Li was supported in part by the National Natural Science Foundation of China (NSFC) under Grant No. 11401464, the China Postdoctoral Science Foundation (No. 2014M560785), and the International Exchange Funds for the Central Universities No. 2014gjhz07.

Author information

Authors and Affiliations

  1. Université de Lyon, CNRS, Ecole Centrale de Lyon, LIRIS, 69134, Lyon, France

    Yinhang Tang & Liming Chen

  2. School of Mathematics and Statistics, Xi’an Jiaotong University, No.28, Xianning West Road, Xi’an, Shaanxi, 710049, P.R. China

    Huibin Li

  3. King Abdullah University of Science and Technologh, V.C.C. Research Center, Thuwal, 23955-6900, Saudi Arabia

    Xiang Sun & Jean-Marie Morvan

  4. Université de Lyon, CNRS, Université Claude Bernard Lyon 1, ICJ UMR 5208, 69622, Villeurbanne, France

    Jean-Marie Morvan

Authors
  1. Yinhang Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Huibin Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xiang Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Jean-Marie Morvan
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Liming Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huibin Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Y., Li, H., Sun, X. et al. Principal Curvature Measures Estimation and Application to 3D Face Recognition. J Math Imaging Vis 59, 211–233 (2017). https://doi.org/10.1007/s10851-017-0728-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-017-0728-2

Keywords

Search

Navigation