Skip to main content
Top
Published in: Soft Computing 14/2021

05-06-2021 | Data analytics and machine learning

Fusion facial semantic feature and incremental learning mechanism for efficient face recognition

Authors: Rui Zhong, Huaiyu Wu, Zhihuan Chen, Qi Zhong

Published in: Soft Computing | Issue 14/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Efficient face recognition can realize fast and accurate face recognition and make it widely used in essential fields such as human–computer interaction and access control. At present, there are many face recognition methods whose recognition rate can reach high accuracy, but the training of the model and the recognition of samples take much time, which leads to insufficient real-time performance. This paper designs a fusion facial semantic feature (FFSF) and an incremental learning mechanism (ILM) for efficient face recognition. FFSF feature is a fusion of facial contour features and facial semantic component features, which can extract contour features and interior features of facial organs (eyes, mouth, nose, and eyebrow) according to facial organs’ position. FFSF features can ensure that the extracted features are concentrated in the face’s most discriminative region, making the extracted features have good discriminative characteristics. Then, we use a clustering algorithm to construct a hierarchical incremental learning tree (HIL-Tree) with a hierarchical structure and use the HIL-Tree to implement the ILM. ILM achieves fast and accurate sample classification by retrieving the nodes in HIL-Tree, and the training samples can be directly added to the HIL-Tree by retrieval instead of rebuilding the HIL-Tree during the training process. Extensive experiments on several public data sets demonstrate the proposed efficient face recognition method’s excellent accuracy and efficiency.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Literature
go back to reference Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041MATHCrossRef Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041MATHCrossRef
go back to reference Bahroun S, Abed R, Zagrouba E (2021) KS-FQA: keyframe selection based on face quality assessment for efficient face recognition in video. IET Image Proc 15:77–90CrossRef Bahroun S, Abed R, Zagrouba E (2021) KS-FQA: keyframe selection based on face quality assessment for efficient face recognition in video. IET Image Proc 15:77–90CrossRef
go back to reference Bashbaghi S, Granger E, Sabourin R, Bilodeau GA (2017) Dynamic ensembles of exemplar-SVMs for still-to-video face recognition. Pattern Recogn 69:61–81CrossRef Bashbaghi S, Granger E, Sabourin R, Bilodeau GA (2017) Dynamic ensembles of exemplar-SVMs for still-to-video face recognition. Pattern Recogn 69:61–81CrossRef
go back to reference Battaglia F, Iannizzotto G, Bello LL (2017) A person authentication system based on RFID tags and a cascade of face recognition algorithms. IEEE Trans Circuits Syst Video Technol 27(8):1676–1690CrossRef Battaglia F, Iannizzotto G, Bello LL (2017) A person authentication system based on RFID tags and a cascade of face recognition algorithms. IEEE Trans Circuits Syst Video Technol 27(8):1676–1690CrossRef
go back to reference Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef
go back to reference Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 3025–3032 Chen D, Cao X, Wen F, Sun J (2013) Blessing of dimensionality: high-dimensional feature and its efficient compression for face verification. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 3025–3032
go back to reference Cho M, Jeong Y (2017) Face recognition performance comparison between fake faces and live faces. Soft Comput 21:3429–3437CrossRef Cho M, Jeong Y (2017) Face recognition performance comparison between fake faces and live faces. Soft Comput 21:3429–3437CrossRef
go back to reference Cho H, Roberts R, Jung B, Choi O, Moo S (2014) An efficient hybrid face recognition algorithm using PCA and GABOR wavelets. Int J Adv Robot Syst 11(1):1–8 Cho H, Roberts R, Jung B, Choi O, Moo S (2014) An efficient hybrid face recognition algorithm using PCA and GABOR wavelets. Int J Adv Robot Syst 11(1):1–8
go back to reference Choi JY, Lee B (2020) Ensemble of deep convolutional neural networks with gabor face representations for face recognition. IEEE Trans Image Process 29:3270–3281CrossRef Choi JY, Lee B (2020) Ensemble of deep convolutional neural networks with gabor face representations for face recognition. IEEE Trans Image Process 29:3270–3281CrossRef
go back to reference Dhekane M, Seal A, Khanna P (2017) Illumination and expression invariant face recognition. Int J Pattern Recogn Artif Intell 31(12):1–15CrossRef Dhekane M, Seal A, Khanna P (2017) Illumination and expression invariant face recognition. Int J Pattern Recogn Artif Intell 31(12):1–15CrossRef
go back to reference Du GY, Tian SL, Qiu YY, Xu CY (2016) Effective and efficient Grassfinch kernel for SVM classification and its application to recognition based on image set. Chaos Solitons Fract 89:295–303MATHCrossRef Du GY, Tian SL, Qiu YY, Xu CY (2016) Effective and efficient Grassfinch kernel for SVM classification and its application to recognition based on image set. Chaos Solitons Fract 89:295–303MATHCrossRef
go back to reference Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40(5):1139–1153CrossRef Duan Y, Lu J, Feng J, Zhou J (2018) Context-aware local binary feature learning for face recognition. IEEE Trans Pattern Anal Mach Intell 40(5):1139–1153CrossRef
go back to reference Feng X, Pietikainen M, Hadid A (2007) Facial expression recognition based on local binary patterns. Pattern Recognit Image Anal 17(4):592–598CrossRef Feng X, Pietikainen M, Hadid A (2007) Facial expression recognition based on local binary patterns. Pattern Recognit Image Anal 17(4):592–598CrossRef
go back to reference Georghiadesa AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRef Georghiadesa AS, Belhumeur PN, Kriegman DJ (2001) From few to many: illumination cone models for face recognition under variable lighting and pose. IEEE Trans Pattern Anal Mach Intell 23(6):643–660CrossRef
go back to reference Guo C, Liang J, Zhan G, Liu Z, Pietikäinen M, Liu L (2019) Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition. IEEE Access 7:174517–174530CrossRef Guo C, Liang J, Zhan G, Liu Z, Pietikäinen M, Liu L (2019) Extended local binary patterns for efficient and robust spontaneous facial micro-expression recognition. IEEE Access 7:174517–174530CrossRef
go back to reference He XJ, Dai BQ (2016) A new traffic signs classification approach based on local and global features extraction. In: Proceeding of the international conference on information communication and management, Hatfield, pp 121–125 He XJ, Dai BQ (2016) A new traffic signs classification approach based on local and global features extraction. In: Proceeding of the international conference on information communication and management, Hatfield, pp 121–125
go back to reference Heikkil M, Pietik M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recognit 42(3):425–436CrossRef Heikkil M, Pietik M, Schmid C (2009) Description of interest regions with local binary patterns. Pattern Recognit 42(3):425–436CrossRef
go back to reference Hou J, Gao H, Xia Q, Qi N (2015) Feature combination and the kNN framework in object classification. IEEE Trans Neural Netw Learn Syst 27(6):1368–1378CrossRef Hou J, Gao H, Xia Q, Qi N (2015) Feature combination and the kNN framework in object classification. IEEE Trans Neural Netw Learn Syst 27(6):1368–1378CrossRef
go back to reference Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Int J Comput vis 96(3):277–279 Huang GB, Ramesh M, Berg T, Learned-Miller E (2007) Labeled faces in the wild: a database for studying face recognition in unconstrained environments. Int J Comput vis 96(3):277–279
go back to reference Huang P, Li T, Gao GW, Yang G (2019) Feature extraction based on graph discriminant embedding and its applications to face recognition. Soft Comput 23(16):7015–7028CrossRef Huang P, Li T, Gao GW, Yang G (2019) Feature extraction based on graph discriminant embedding and its applications to face recognition. Soft Comput 23(16):7015–7028CrossRef
go back to reference Jia H, Martinez AM (2009) Support vector machines in face recognition with occlusions. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Miami, pp 136–141 Jia H, Martinez AM (2009) Support vector machines in face recognition with occlusions. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Miami, pp 136–141
go back to reference Karczmarek P, Pedrycz W, Kiersztyn A, Rutka P (2017) A study in facial features saliency in face recognition: an analytic hierarchy process approach. Soft Comput 21(24):7503–7517CrossRef Karczmarek P, Pedrycz W, Kiersztyn A, Rutka P (2017) A study in facial features saliency in face recognition: an analytic hierarchy process approach. Soft Comput 21(24):7503–7517CrossRef
go back to reference Lei Z, Pietikainen M, Li SZ (2013) Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell 36(2):289–302 Lei Z, Pietikainen M, Li SZ (2013) Learning discriminant face descriptor. IEEE Trans Pattern Anal Mach Intell 36(2):289–302
go back to reference Liang J, Wang M, Chai Z, Wu Q (2014) Different lighting processing and feature extraction methods for efficient face recognition. IET Image Proc 8(9):528–538CrossRef Liang J, Wang M, Chai Z, Wu Q (2014) Different lighting processing and feature extraction methods for efficient face recognition. IET Image Proc 8(9):528–538CrossRef
go back to reference Liang J, Tu H, Liu F, Zhao Q, Jain A (2020) 3D face reconstruction from mugshots: Application to arbitrary view face recognition. Neurocomputing 410(14):12–27CrossRef Liang J, Tu H, Liu F, Zhao Q, Jain A (2020) 3D face reconstruction from mugshots: Application to arbitrary view face recognition. Neurocomputing 410(14):12–27CrossRef
go back to reference Liao S, Law MWK, Chung ACS (2009) Dominant local binary patterns for texture classification. IEEE Trans Image Process 18(5):1107–1118MathSciNetMATHCrossRef Liao S, Law MWK, Chung ACS (2009) Dominant local binary patterns for texture classification. IEEE Trans Image Process 18(5):1107–1118MathSciNetMATHCrossRef
go back to reference Liu QF, Liu CJ (2017) A novel locally linear KNN method with applications to visual recognition. IEEE Trans Neural Netw Learn Syst 28(9):2010–2021MathSciNetCrossRef Liu QF, Liu CJ (2017) A novel locally linear KNN method with applications to visual recognition. IEEE Trans Neural Netw Learn Syst 28(9):2010–2021MathSciNetCrossRef
go back to reference Liu L, Zhao L, Long Y, Kuang G, Fieguth P (2012) Extended local binary patterns for texture classification. Image Vis Comput 30(2):86–99CrossRef Liu L, Zhao L, Long Y, Kuang G, Fieguth P (2012) Extended local binary patterns for texture classification. Image Vis Comput 30(2):86–99CrossRef
go back to reference Liu R, Feng WG, Zhu M (2013) Expression and lighting invariant face recognition using fast tree-based matching. Electron Lett 49(22):1379–1381CrossRef Liu R, Feng WG, Zhu M (2013) Expression and lighting invariant face recognition using fast tree-based matching. Electron Lett 49(22):1379–1381CrossRef
go back to reference Liu F, Zhao Q, Liu X, Zeng D (2020) Joint face alignment and 3D face reconstruction with application to face recognition. IEEE Trans Pattern Anal Mach Intell 42(3):664–678CrossRef Liu F, Zhao Q, Liu X, Zeng D (2020) Joint face alignment and 3D face reconstruction with application to face recognition. IEEE Trans Pattern Anal Mach Intell 42(3):664–678CrossRef
go back to reference Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056CrossRef Lu J, Liong VE, Zhou X, Zhou J (2015) Learning compact binary face descriptor for face recognition. IEEE Trans Pattern Anal Mach Intell 37(10):2041–2056CrossRef
go back to reference Lu J, Liong VE, Zhou J (2018) Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 40(8):1979–1993CrossRef Lu J, Liong VE, Zhou J (2018) Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 40(8):1979–1993CrossRef
go back to reference Macqueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley symposium on mathematical statistics and probability. University of California, pp 281–297. Macqueen J (1967) Some methods for classification and analysis of multivariate observations. In: Proceedings of the 5th Berkeley symposium on mathematical statistics and probability. University of California, pp 281–297.
go back to reference Mahmood Z, Ali T, Khan SU (2016) Effects of pose and image resolution on automatic face recognition. IET Biom 5(2):111–119CrossRef Mahmood Z, Ali T, Khan SU (2016) Effects of pose and image resolution on automatic face recognition. IET Biom 5(2):111–119CrossRef
go back to reference Martinez AM, Benavente R (1998) The AR face database. CVC technical report 24 Martinez AM, Benavente R (1998) The AR face database. CVC technical report 24
go back to reference Roh SB, Oh SK, Yoon JH, Seo K (2019) Design of face recognition system based on fuzzy transform and radial basis function neural networks. Soft Comput 23(13):4969–4985CrossRef Roh SB, Oh SK, Yoon JH, Seo K (2019) Design of face recognition system based on fuzzy transform and radial basis function neural networks. Soft Comput 23(13):4969–4985CrossRef
go back to reference Sim T, Baker S, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618CrossRef Sim T, Baker S, Bsat M (2003) The CMU pose, illumination, and expression database. IEEE Trans Pattern Anal Mach Intell 25(12):1615–1618CrossRef
go back to reference Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Proceeding of the IEEE conference on computer vision and pattern recognition,Columbus, pp 1891–1898 Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In: Proceeding of the IEEE conference on computer vision and pattern recognition,Columbus, pp 1891–1898
go back to reference Tan XY, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetMATHCrossRef Tan XY, Triggs B (2010) Enhanced local texture feature sets for face recognition under difficult lighting conditions. IEEE Trans Image Process 19(6):1635–1650MathSciNetMATHCrossRef
go back to reference Tan H, Yang B, Ma M (2014) Face recognition based on the fusion of global and local HOG features of face images. IET Comput Vis 8(3):224–234CrossRef Tan H, Yang B, Ma M (2014) Face recognition based on the fusion of global and local HOG features of face images. IET Comput Vis 8(3):224–234CrossRef
go back to reference Tang H, Yin B, Sun Y, Hu Y (2013) 3D face recognition using local binary patterns. Signal Process 93(8):2190–2198CrossRef Tang H, Yin B, Sun Y, Hu Y (2013) 3D face recognition using local binary patterns. Signal Process 93(8):2190–2198CrossRef
go back to reference Vu NS, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365MathSciNetMATHCrossRef Vu NS, Caplier A (2012) Enhanced patterns of oriented edge magnitudes for face recognition and image matching. IEEE Trans Image Process 21(3):1352–1365MathSciNetMATHCrossRef
go back to reference Weng JY, Hwang WS (2007) Incremental hierarchical discriminant regression. IEEE Trans Neural Netw 18(2):397–415CrossRef Weng JY, Hwang WS (2007) Incremental hierarchical discriminant regression. IEEE Trans Neural Netw 18(2):397–415CrossRef
go back to reference Xiong X, Torre FD (2013) Supervised descent method and its application to face alignment. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 532–539 Xiong X, Torre FD (2013) Supervised descent method and its application to face alignment. In: Proceeding of the IEEE conference on computer vision and pattern recognition, Portland, pp 532–539
go back to reference Xu J, Xie S, Zhu W (2017) Marginal patch alignment for dimensionality reduction. Soft Comput 21:2347–2356CrossRef Xu J, Xie S, Zhu W (2017) Marginal patch alignment for dimensionality reduction. Soft Comput 21:2347–2356CrossRef
go back to reference Yang F, Mao KZ, Lee GK, Tang W (2015) Emphasizing minority class in LDA for feature subset selection on high-dimensional small-sized problems. IEEE Trans Knowl Data Eng 27(1):88–101CrossRef Yang F, Mao KZ, Lee GK, Tang W (2015) Emphasizing minority class in LDA for feature subset selection on high-dimensional small-sized problems. IEEE Trans Knowl Data Eng 27(1):88–101CrossRef
go back to reference Yang WK, Wang ZY, Zhang BC (2016) Face recognition using adaptive local ternary patterns method. Neurocomputing 213:183–190CrossRef Yang WK, Wang ZY, Zhang BC (2016) Face recognition using adaptive local ternary patterns method. Neurocomputing 213:183–190CrossRef
go back to reference Zhao J, Han J, Shao L (2018) Unconstrained face recognition using a set-to-set distance measure on deep learned features. IEEE Trans Circuits Syst Video Technol 28(10):2679–2689CrossRef Zhao J, Han J, Shao L (2018) Unconstrained face recognition using a set-to-set distance measure on deep learned features. IEEE Trans Circuits Syst Video Technol 28(10):2679–2689CrossRef
go back to reference Zhao J, Xiong L, Li J, Xing J, Yan S, Feng J (2019) 3D-aided dual-agent GANs for unconstrained face recognition. IEEE Trans Pattern Anal Mach Intell 41(10):2380–2394CrossRef Zhao J, Xiong L, Li J, Xing J, Yan S, Feng J (2019) 3D-aided dual-agent GANs for unconstrained face recognition. IEEE Trans Pattern Anal Mach Intell 41(10):2380–2394CrossRef
go back to reference Zheng J, Ranjan R, Chen C, Chen J, Castillo CD, Chellappa R (2020) An automatic system for unconstrained video-based face recognition. IEEE Trans Biom Behav Identity Sci 2(3):194–209CrossRef Zheng J, Ranjan R, Chen C, Chen J, Castillo CD, Chellappa R (2020) An automatic system for unconstrained video-based face recognition. IEEE Trans Biom Behav Identity Sci 2(3):194–209CrossRef
go back to reference Zhong Y, Deng W, Hu J, Zhao D, Li X, Wen D (2020) SFace: sigmoid-constrained hypersphere loss for robust face recognition. IEEE Trans Image Process 30:2587-2598CrossRef Zhong Y, Deng W, Hu J, Zhao D, Li X, Wen D (2020) SFace: sigmoid-constrained hypersphere loss for robust face recognition. IEEE Trans Image Process 30:2587-2598CrossRef
go back to reference Zhu WJ, Yan YH, Peng YS (2017) Pair of projections based on sparse consistence with applications to efficient face recognition. Signal Process Image Commun 55:32–40CrossRef Zhu WJ, Yan YH, Peng YS (2017) Pair of projections based on sparse consistence with applications to efficient face recognition. Signal Process Image Commun 55:32–40CrossRef
Metadata
Title
Fusion facial semantic feature and incremental learning mechanism for efficient face recognition
Authors
Rui Zhong
Huaiyu Wu
Zhihuan Chen
Qi Zhong
Publication date
05-06-2021
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 14/2021
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-021-05915-x

Other articles of this Issue 14/2021

Soft Computing 14/2021 Go to the issue

Data analytics and machine learning

Racing trees to query partial data

Premium Partner