Skip to main content
Erschienen in: Neural Processing Letters 2/2019

26.10.2018

Fine Tuning Dual Streams Deep Network with Multi-scale Pyramid Decision for Heterogeneous Face Recognition

verfasst von: Weipeng Hu, Haifeng Hu

Erschienen in: Neural Processing Letters | Ausgabe 2/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we propose a novel method called fine tuning dual streams deep network (FTDSDN) with multi-scale pyramid decision (MsPD) for solving heterogeneous face recognition task. As an extension of classical CNNs, FTDSDN can remove highly non-linear modality information and reserve the discriminative information using Rayleigh quotient objective function. Furthermore, we develop a powerful joint decision strategy called MsPD to adaptively adjust the weight of sub structure and obtain more robust classification performance. Experimental results show our proposed method achieves better performance on the challenging CASIA NIR-VIS 2.0 database, the heterogeneous face biometrics database, the CUHK face sketch FERET database, and the CUHK face sketch database, which demonstrates the effectiveness of our proposed approach.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Alex AT, Asari VK, Mathew A (2013) Local difference of Gaussian binary pattern: robust features for face sketch recognition. In: IEEE international conference on systems, man, and cybernetics, pp 1211–1216 Alex AT, Asari VK, Mathew A (2013) Local difference of Gaussian binary pattern: robust features for face sketch recognition. In: IEEE international conference on systems, man, and cybernetics, pp 1211–1216
2.
Zurück zum Zitat Ding C, Tao D (2018) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014CrossRef Ding C, Tao D (2018) Trunk-branch ensemble convolutional neural networks for video-based face recognition. IEEE Trans Pattern Anal Mach Intell 40(4):1002–1014CrossRef
3.
Zurück zum Zitat Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Hum Genet 7(2):179–188 Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Hum Genet 7(2):179–188
4.
Zurück zum Zitat Fukunaga K (2013) Introduction to statistical pattern recognition. Academic press, CambridgeMATH Fukunaga K (2013) Introduction to statistical pattern recognition. Academic press, CambridgeMATH
5.
Zurück zum Zitat Gong D, Li Z, Huang W, Li X, Tao D (2017) Heterogeneous face recognition: a common encoding feature discriminant approach. IEEE Trans Image Process 26(5):2079–2089MathSciNetCrossRef Gong D, Li Z, Huang W, Li X, Tao D (2017) Heterogeneous face recognition: a common encoding feature discriminant approach. IEEE Trans Image Process 26(5):2079–2089MathSciNetCrossRef
6.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: IEEE conference computer vision and pattern recognition, pp 770–778
7.
Zurück zum Zitat He R, Wu X, Sun Z, Tan T (2017) Learning invariant deep representation for NIR-VIS face recognition. In: AAAI conference on artificial intelligence, vol 4, p 7 He R, Wu X, Sun Z, Tan T (2017) Learning invariant deep representation for NIR-VIS face recognition. In: AAAI conference on artificial intelligence, vol 4, p 7
8.
Zurück zum Zitat Hou CA, Yang M, Wang YCF (2014) Domain adaptive self-taught learning for heterogeneous face recognition. In: International conference on pattern recognition. IEEE, pp 3068–3073 Hou CA, Yang M, Wang YCF (2014) Domain adaptive self-taught learning for heterogeneous face recognition. In: International conference on pattern recognition. IEEE, pp 3068–3073
9.
Zurück zum Zitat Hu S, Short N, Riggan BS, Chasse M, Sarfraz MS (2017) Heterogeneous face recognition: recent advances in infrared-to-visible matching. In: IEEE international conference on automatic face and gesture recognition, pp 883–890 Hu S, Short N, Riggan BS, Chasse M, Sarfraz MS (2017) Heterogeneous face recognition: recent advances in infrared-to-visible matching. In: IEEE international conference on automatic face and gesture recognition, pp 883–890
10.
Zurück zum Zitat Huang X, Lei Z, Fan M, Wang X, Li SZ (2013) Regularized discriminative spectral regression method for heterogeneous face matching. IEEE Trans Image Process 22(1):353–362MathSciNetCrossRef Huang X, Lei Z, Fan M, Wang X, Li SZ (2013) Regularized discriminative spectral regression method for heterogeneous face matching. IEEE Trans Image Process 22(1):353–362MathSciNetCrossRef
11.
Zurück zum Zitat Huo J, Gao Y, Shi Y, Yang W, Yin H (2018) Heterogeneous face recognition by margin-based cross-modality metric learning. IEEE Trans Cybern 48(6):1814–1826CrossRef Huo J, Gao Y, Shi Y, Yang W, Yin H (2018) Heterogeneous face recognition by margin-based cross-modality metric learning. IEEE Trans Cybern 48(6):1814–1826CrossRef
12.
Zurück zum Zitat Jin Y, Lu J, Ruan Q (2015) Large margin coupled feature learning for cross-modal face recognition. In: International conference on biometrics. IEEE, pp 286–292 Jin Y, Lu J, Ruan Q (2015) Large margin coupled feature learning for cross-modal face recognition. In: International conference on biometrics. IEEE, pp 286–292
13.
Zurück zum Zitat Kan M, Shan S, Chen X (2016) Multi-view deep network for cross-view classification. In: IEEE conference computer vision and pattern recognition, pp 4847–4855 Kan M, Shan S, Chen X (2016) Multi-view deep network for cross-view classification. In: IEEE conference computer vision and pattern recognition, pp 4847–4855
14.
Zurück zum Zitat Kan M, Shan S, Zhang H, Lao S, Chen X (2016) Multi-view discriminant analysis. IEEE Trans Pattern Anal Mach Intell 38(1):188–194CrossRef Kan M, Shan S, Zhang H, Lao S, Chen X (2016) Multi-view discriminant analysis. IEEE Trans Pattern Anal Mach Intell 38(1):188–194CrossRef
15.
Zurück zum Zitat Karlpearson FRS (1901) LIII. On lines and planes of closest fit to systems of points in space. Philos Mag 2(11):559–572CrossRef Karlpearson FRS (1901) LIII. On lines and planes of closest fit to systems of points in space. Philos Mag 2(11):559–572CrossRef
16.
Zurück zum Zitat Klare BF, Li Z, Jain AK (2011) Matching forensic sketches to mug shot photos. IEEE Trans Pattern Anal Mach Intell 33(3):639CrossRef Klare BF, Li Z, Jain AK (2011) Matching forensic sketches to mug shot photos. IEEE Trans Pattern Anal Mach Intell 33(3):639CrossRef
17.
Zurück zum Zitat Lei Z, Li SZ (2009) Coupled spectral regression for matching heterogeneous faces. In: IEEE conference computer vision and pattern recognition, pp 1123–1128 Lei Z, Li SZ (2009) Coupled spectral regression for matching heterogeneous faces. In: IEEE conference computer vision and pattern recognition, pp 1123–1128
18.
Zurück zum Zitat Lezama J, Qiu Q, Sapiro G (2017) Not afraid of the dark: NIR-VIS face recognition via cross-spectral hallucination and low-rank embedding. In: IEEE conference computer vision and pattern recognition, pp 6807–6816 Lezama J, Qiu Q, Sapiro G (2017) Not afraid of the dark: NIR-VIS face recognition via cross-spectral hallucination and low-rank embedding. In: IEEE conference computer vision and pattern recognition, pp 6807–6816
19.
Zurück zum Zitat Li J, Hao P, Zhang C, Dou M (2008) Hallucinating faces from thermal infrared images. In: IEEE international conference on image processing, pp 465–468 Li J, Hao P, Zhang C, Dou M (2008) Hallucinating faces from thermal infrared images. In: IEEE international conference on image processing, pp 465–468
20.
Zurück zum Zitat Li S, Yi D, Lei Z, Liao S (2013) The CASIA NIR-VIS 2.0 face database. In: IEEE conference computer vision and pattern recognition workshops, pp 348–353 Li S, Yi D, Lei Z, Liao S (2013) The CASIA NIR-VIS 2.0 face database. In: IEEE conference computer vision and pattern recognition workshops, pp 348–353
21.
Zurück zum Zitat Li SZ, Lei Z, Ao M (2009) The HFB face database for heterogeneous face biometrics research. In: IEEE conference computer vision and pattern recognition, pp 1–8 Li SZ, Lei Z, Ao M (2009) The HFB face database for heterogeneous face biometrics research. In: IEEE conference computer vision and pattern recognition, pp 1–8
22.
Zurück zum Zitat Lin D, Tang X (2006) Inter-modality face recognition. In: European conference on computer vision, pp 13–26 Lin D, Tang X (2006) Inter-modality face recognition. In: European conference on computer vision, pp 13–26
23.
Zurück zum Zitat Liu X, Kan M, Wu W, Shan S, Chen X (2017) VIPLFaceNet: an open source deep face recognition SDK. Front Comput Sci 11(2):208–218CrossRef Liu X, Kan M, Wu W, Shan S, Chen X (2017) VIPLFaceNet: an open source deep face recognition SDK. Front Comput Sci 11(2):208–218CrossRef
24.
Zurück zum Zitat Liu X, Song L, Wu X, Tan T (2016) Transferring deep representation for NIR-VIS heterogeneous face recognition. In: International conference on biometrics, pp 1–8 Liu X, Song L, Wu X, Tan T (2016) Transferring deep representation for NIR-VIS heterogeneous face recognition. In: International conference on biometrics, pp 1–8
25.
Zurück zum Zitat Lu J, Erin LV, Zhou J (2017) Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell PP(99):1–1 Lu J, Erin LV, Zhou J (2017) Simultaneous local binary feature learning and encoding for homogeneous and heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell PP(99):1–1
26.
Zurück zum Zitat Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: International conference on international conference on machine learning, pp 807–814 Nair V, Hinton GE (2010) Rectified linear units improve restricted Boltzmann machines. In: International conference on international conference on machine learning, pp 807–814
27.
Zurück zum Zitat Parkhi OM, Vedaldi A, Zisserman A, et al (2015) Deep face recognition. In: British machine vision conference, vol 1, p 6 Parkhi OM, Vedaldi A, Zisserman A, et al (2015) Deep face recognition. In: British machine vision conference, vol 1, p 6
28.
Zurück zum Zitat Peng C, Gao X, Wang N, Li J (2017) Graphical representation for heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 39(2):301–312CrossRef Peng C, Gao X, Wang N, Li J (2017) Graphical representation for heterogeneous face recognition. IEEE Trans Pattern Anal Mach Intell 39(2):301–312CrossRef
29.
Zurück zum Zitat Reale C, Lee H, Kwon H (2017) Deep heterogeneous face recognition networks based on cross-modal distillation and an equitable distance metric. In: IEEE conference computer vision and pattern recognition workshops, pp 32–38 Reale C, Lee H, Kwon H (2017) Deep heterogeneous face recognition networks based on cross-modal distillation and an equitable distance metric. In: IEEE conference computer vision and pattern recognition workshops, pp 32–38
30.
Zurück zum Zitat Saxena S, Verbeek J (2016) Heterogeneous face recognition with CNNs. In: European conference on computer vision, pp 483–491 Saxena S, Verbeek J (2016) Heterogeneous face recognition with CNNs. In: European conference on computer vision, pp 483–491
31.
Zurück zum Zitat Sharma A, Jacobs DW (2011) Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. In: IEEE conference computer vision and pattern recognition, pp 593–600 Sharma A, Jacobs DW (2011) Bypassing synthesis: PLS for face recognition with pose, low-resolution and sketch. In: IEEE conference computer vision and pattern recognition, pp 593–600
32.
Zurück zum Zitat Shi H, Wang X, Yi D, Lei Z, Zhu X, Li SZ (2017) Cross-modality face recognition via heterogeneous joint bayesian. IEEE Signal Process Lett 24(1):81–85CrossRef Shi H, Wang X, Yi D, Lei Z, Zhu X, Li SZ (2017) Cross-modality face recognition via heterogeneous joint bayesian. IEEE Signal Process Lett 24(1):81–85CrossRef
33.
Zurück zum Zitat Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. In: Advances in neural information processing systems, pp 568–576 Simonyan K, Zisserman A (2014) Two-stream convolutional networks for action recognition in videos. In: Advances in neural information processing systems, pp 568–576
34.
Zurück zum Zitat Song Y, Bao L, Yang Q, Yang MH (2014) Real-time exemplar-based face sketch synthesis. In: European conference on computer vision. Springer, pp 800–813 Song Y, Bao L, Yang Q, Yang MH (2014) Real-time exemplar-based face sketch synthesis. In: European conference on computer vision. Springer, pp 800–813
35.
Zurück zum Zitat Tenenbaum J (2000) Separating style and content with bilinear models. Neural Comput 12:1247–1283CrossRef Tenenbaum J (2000) Separating style and content with bilinear models. Neural Comput 12:1247–1283CrossRef
36.
Zurück zum Zitat Tian Y, Yan C, Bai X, Zhou J (2017) Heterogeneous face recognition via Grassmannian based nearest subspace search. In: IEEE international conference on image processing, pp 1077–1081 Tian Y, Yan C, Bai X, Zhou J (2017) Heterogeneous face recognition via Grassmannian based nearest subspace search. In: IEEE international conference on image processing, pp 1077–1081
37.
Zurück zum Zitat Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30CrossRef Wang N, Tao D, Gao X, Li X, Li J (2014) A comprehensive survey to face hallucination. Int J Comput Vis 106(1):9–30CrossRef
38.
Zurück zum Zitat Wang S, Huang D, Wang Y, Tang Y (2017) 2D–3D heterogeneous face recognition based on deep canonical correlation analysis. In: Chinese conference on biometric recognition. Springer, pp 77–85 Wang S, Huang D, Wang Y, Tang Y (2017) 2D–3D heterogeneous face recognition based on deep canonical correlation analysis. In: Chinese conference on biometric recognition. Springer, pp 77–85
39.
Zurück zum Zitat Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: IEEE conference computer vision and pattern recognition, pp 2216–2223 Wang S, Zhang L, Liang Y, Pan Q (2012) Semi-coupled dictionary learning with applications to image super-resolution and photo-sketch synthesis. In: IEEE conference computer vision and pattern recognition, pp 2216–2223
40.
Zurück zum Zitat Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967CrossRef Wang X, Tang X (2009) Face photo-sketch synthesis and recognition. IEEE Trans Pattern Anal Mach Intell 31(11):1955–1967CrossRef
41.
Zurück zum Zitat Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision. Springer, pp 499–515 Wen Y, Zhang K, Li Z, Qiao Y (2016) A discriminative feature learning approach for deep face recognition. In: European conference on computer vision. Springer, pp 499–515
42.
43.
Zurück zum Zitat Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2006) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40CrossRef Yan S, Xu D, Zhang B, Zhang HJ, Yang Q, Lin S (2006) Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40CrossRef
44.
Zurück zum Zitat Yi D, Lei Z, Li SZ (2015) Shared representation learning for heterogenous face recognition. In: IEEE international conference and workshops on automatic face and gesture recognition, vol 1, pp 1–7 Yi D, Lei Z, Li SZ (2015) Shared representation learning for heterogenous face recognition. In: IEEE international conference and workshops on automatic face and gesture recognition, vol 1, pp 1–7
46.
Zurück zum Zitat Yi D, Liu R, Chu R, Lei Z, Li SZ (2007) Face matching between near infrared and visible light images. In: International conference on biometrics. Springer, pp 523–530 Yi D, Liu R, Chu R, Lei Z, Li SZ (2007) Face matching between near infrared and visible light images. In: International conference on biometrics. Springer, pp 523–530
47.
Zurück zum Zitat Zhang W, Shu Z, Samaras D, Chen L (2017) Improving heterogeneous face recognition with conditional adversarial networks. arXiv preprint arXiv:1709.02848 Zhang W, Shu Z, Samaras D, Chen L (2017) Improving heterogeneous face recognition with conditional adversarial networks. arXiv preprint arXiv:​1709.​02848
48.
Zurück zum Zitat Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: IEEE conference computer vision and pattern recognition, pp 513–520 Zhang W, Wang X, Tang X (2011) Coupled information-theoretic encoding for face photo-sketch recognition. In: IEEE conference computer vision and pattern recognition, pp 513–520
49.
Zurück zum Zitat Zhong J, Gao X, Tian C (2007) Face sketch synthesis using E-HMM and selective ensemble. In: IEEE international conference on acoustics, speech and signal processing, pp 485–488 Zhong J, Gao X, Tian C (2007) Face sketch synthesis using E-HMM and selective ensemble. In: IEEE international conference on acoustics, speech and signal processing, pp 485–488
Metadaten
Titel
Fine Tuning Dual Streams Deep Network with Multi-scale Pyramid Decision for Heterogeneous Face Recognition
verfasst von
Weipeng Hu
Haifeng Hu
Publikationsdatum
26.10.2018
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 2/2019
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-018-9942-1

Weitere Artikel der Ausgabe 2/2019

Neural Processing Letters 2/2019 Zur Ausgabe

Neuer Inhalt