Skip to main content
Top
Published in: Neural Computing and Applications 3/2017

24-10-2015 | Original Article

Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning

Authors: Fei Wu, Xiao-Yuan Jing, Qian Liu, Song-Song Wu, Guo-Liang He

Published in: Neural Computing and Applications | Issue 3/2017

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Kernel discriminant subspace learning technique is effective to exploit the structure of image dataset in the high-dimensional nonlinear space. However, for large-scale image recognition applications, this technique usually suffers from large computational burden. Although some kernel accelerating methods have been presented, how to greatly reduce computing time and simultaneously keep favorable recognition accuracy is still challenging. In this paper, we introduce the idea of parallel computing into kernel subspace learning and build a parallel kernel discriminant subspace learning framework. In this framework, we firstly design a random non-overlapping equal data division strategy to divide the whole training set into several subsets and assign each computational node a subset. Then, we separately learn kernel discriminant subspaces from these subsets without mutual communications and finally select the most appropriate subspace to classify test samples. Under the built framework, we propose two novel kernel subspace learning approaches, i.e., parallel kernel discriminant analysis (PKDA) and parallel kernel semi-supervised discriminant analysis (PKSDA). We show the superiority of the proposed approaches in terms of time complexity as compared with related methods, and provide the fundamental supports for our framework. For experiment, we establish a parallel computing environment and employ three public large-scale image databases as experiment data. Experimental results demonstrate the efficiency and effectiveness of the proposed approaches.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Belhumeur PN, Hespanda J, Kiregeman D (1997) Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef Belhumeur PN, Hespanda J, Kiregeman D (1997) Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef
3.
go back to reference Pang SN, Ban T, Kadobayashi Y, Kasabov NK (2012) LDA merging and splitting with applications to multiagent cooperative learning and system alteration. IEEE Trans Syst Man Cybern Part B 42(2):552–564CrossRef Pang SN, Ban T, Kadobayashi Y, Kasabov NK (2012) LDA merging and splitting with applications to multiagent cooperative learning and system alteration. IEEE Trans Syst Man Cybern Part B 42(2):552–564CrossRef
4.
go back to reference Ye JP (2007) Least squares linear discriminant analysis. In: International conference on machine learning, pp 1087–1093 Ye JP (2007) Least squares linear discriminant analysis. In: International conference on machine learning, pp 1087–1093
5.
go back to reference Su Y, Shan SG, Chen XL, Gao W (2008) Classifiability-based optimal discriminatory projection pursuit. In: IEEE conference on computer vision and pattern recognition, pp 1–7 Su Y, Shan SG, Chen XL, Gao W (2008) Classifiability-based optimal discriminatory projection pursuit. In: IEEE conference on computer vision and pattern recognition, pp 1–7
6.
go back to reference Zhang TH, Huang KQ, Li XL, Yang J, Tao DC (2010) Discriminative orthogonal neighborhood-preserving projections for classification. IEEE Trans Syst Man Cybern Part B 40(1):253–263CrossRef Zhang TH, Huang KQ, Li XL, Yang J, Tao DC (2010) Discriminative orthogonal neighborhood-preserving projections for classification. IEEE Trans Syst Man Cybern Part B 40(1):253–263CrossRef
7.
go back to reference Li X, Hu WM, Wang HZ, Zhang ZF (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579CrossRef Li X, Hu WM, Wang HZ, Zhang ZF (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579CrossRef
8.
go back to reference Zhao C, Miao D, Lai Z, Gao C, Liu C, Yang J (2013) Two-dimensional color uncorrelated discriminant analysis for face recognition. Neurocomputing 113:251–261CrossRef Zhao C, Miao D, Lai Z, Gao C, Liu C, Yang J (2013) Two-dimensional color uncorrelated discriminant analysis for face recognition. Neurocomputing 113:251–261CrossRef
9.
go back to reference Zhong FJ, Zhang JS (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027MathSciNetCrossRef Zhong FJ, Zhang JS (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027MathSciNetCrossRef
10.
go back to reference Yang M, Sun S (2014) Multi-view uncorrelated linear discriminant analysis with applications to handwritten digit recognition. In: International joint conference on neural networks, pp 4175–4181 Yang M, Sun S (2014) Multi-view uncorrelated linear discriminant analysis with applications to handwritten digit recognition. In: International joint conference on neural networks, pp 4175–4181
11.
go back to reference Zhao C, Lai Z, Miao D, Wei Z, Liu C (2014) Graph embedding discriminant analysis for face recognition. Neural Comput Appl 24(7–8):1697–1706CrossRef Zhao C, Lai Z, Miao D, Wei Z, Liu C (2014) Graph embedding discriminant analysis for face recognition. Neural Comput Appl 24(7–8):1697–1706CrossRef
12.
go back to reference Wang Z, Ruan Q, An G (2015) Projection-optimal local fisher discriminant analysis for feature extraction. Neural Comput Appl 26(3):589–601CrossRef Wang Z, Ruan Q, An G (2015) Projection-optimal local fisher discriminant analysis for feature extraction. Neural Comput Appl 26(3):589–601CrossRef
13.
14.
go back to reference Li S, Fu Y (2014) Robust subspace discovery through supervised low-rank constraints. SIAM international conference on data mining, pp 163–171 Li S, Fu Y (2014) Robust subspace discovery through supervised low-rank constraints. SIAM international conference on data mining, pp 163–171
15.
go back to reference Li S, Fu Y (2015) Learning robust and discriminative subspace with low-rank constraints. IEEE Trans Neural Netw Learn Syst (in press) Li S, Fu Y (2015) Learning robust and discriminative subspace with low-rank constraints. IEEE Trans Neural Netw Learn Syst (in press)
16.
go back to reference Zhang DQ, Zhou ZH, Chen SC (2007) Semi-supervised dimensionality reduction. In: International conference on data mining, pp 629–634 Zhang DQ, Zhou ZH, Chen SC (2007) Semi-supervised dimensionality reduction. In: International conference on data mining, pp 629–634
17.
go back to reference Cai D, He XF, Han JW (2007) Semi-supervised discriminant analysis. In: International conference on computer vision, pp 1–7 Cai D, He XF, Han JW (2007) Semi-supervised discriminant analysis. In: International conference on computer vision, pp 1–7
18.
go back to reference Fan MY, Gu NN, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recognit 44(8):1777–1784CrossRefMATH Fan MY, Gu NN, Qiao H, Zhang B (2011) Sparse regularization for semi-supervised classification. Pattern Recognit 44(8):1777–1784CrossRefMATH
19.
go back to reference Zhang TT, Ji RR, Liu W, Tao DC, Hua G (2013) Semi-supervised learning with manifold fitted graphs. In: International joint conference artificial intelligence, pp 1896–1902 Zhang TT, Ji RR, Liu W, Tao DC, Hua G (2013) Semi-supervised learning with manifold fitted graphs. In: International joint conference artificial intelligence, pp 1896–1902
20.
21.
go back to reference Zhao M, Zhang Z, Chow TW, Li B (2014) A general soft label based linear discriminant analysis for semi-supervised dimensionality reduction. Neural Netw 55:83–97CrossRefMATH Zhao M, Zhang Z, Chow TW, Li B (2014) A general soft label based linear discriminant analysis for semi-supervised dimensionality reduction. Neural Netw 55:83–97CrossRefMATH
22.
go back to reference Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76CrossRef Gao Q, Huang Y, Gao X, Shen W, Zhang H (2015) A novel semi-supervised learning for face recognition. Neurocomputing 152:69–76CrossRef
23.
go back to reference Kim M (2015) Greedy approaches to semi-supervised subspace learning. Pattern Recognit 48(4):1559–1566CrossRef Kim M (2015) Greedy approaches to semi-supervised subspace learning. Pattern Recognit 48(4):1559–1566CrossRef
24.
go back to reference Taylor JS, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, CambridgeCrossRef Taylor JS, Cristianini N (2004) Kernel methods for pattern analysis. Cambridge University Press, CambridgeCrossRef
25.
go back to reference Mika S, Ratsch G, Weston J, Scholkopf B, Muller K (1999) Fisher discriminant analysis with kernels. In: IEEE signal processing society workshop on neural networks for signal processing IX, pp 41–48 Mika S, Ratsch G, Weston J, Scholkopf B, Muller K (1999) Fisher discriminant analysis with kernels. In: IEEE signal processing society workshop on neural networks for signal processing IX, pp 41–48
26.
go back to reference Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404CrossRef Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404CrossRef
27.
go back to reference Jing XY, Yao YF, Zhang D, Yang JY, Li M (2007) Face and palmprint pixel level fusion and KDCV–RBF classifier for small sample biometric recognition. Pattern Recognit 40(11):3209–3224CrossRefMATH Jing XY, Yao YF, Zhang D, Yang JY, Li M (2007) Face and palmprint pixel level fusion and KDCV–RBF classifier for small sample biometric recognition. Pattern Recognit 40(11):3209–3224CrossRefMATH
28.
go back to reference Chen B, Yuan L, Liu H, Bao Z (2007) Kernel subclass discriminant analysis. Neurocomputing 71(1–3):455–458CrossRef Chen B, Yuan L, Liu H, Bao Z (2007) Kernel subclass discriminant analysis. Neurocomputing 71(1–3):455–458CrossRef
29.
go back to reference Zheng WM, Lin ZC, Tang XO (2010) A rank-one update algorithm for fast solving kernel Foley–Sammon optimal discriminant vectors. IEEE Trans Neural Netw 21(3):393–403CrossRef Zheng WM, Lin ZC, Tang XO (2010) A rank-one update algorithm for fast solving kernel Foley–Sammon optimal discriminant vectors. IEEE Trans Neural Netw 21(3):393–403CrossRef
30.
go back to reference Li JB, Peng Y, Liu D (2013) Quasiconformal kernel common locality discriminant analysis with application to breast cancer diagnosis. Inf Sci 223:256–269MathSciNetCrossRefMATH Li JB, Peng Y, Liu D (2013) Quasiconformal kernel common locality discriminant analysis with application to breast cancer diagnosis. Inf Sci 223:256–269MathSciNetCrossRefMATH
31.
go back to reference Zheng W, Lin Z, Wang H (2014) L1-norm kernel discriminant analysis via Bayes error bound optimization for robust feature extraction. IEEE Trans Neural Netw Learn Syst 25(4):793–805CrossRef Zheng W, Lin Z, Wang H (2014) L1-norm kernel discriminant analysis via Bayes error bound optimization for robust feature extraction. IEEE Trans Neural Netw Learn Syst 25(4):793–805CrossRef
32.
go back to reference Iosifidis A, Tefas A, Pitas I (2014) Kernel reference discriminant analysis. Pattern Recognit Lett 49:85–91CrossRef Iosifidis A, Tefas A, Pitas I (2014) Kernel reference discriminant analysis. Pattern Recognit Lett 49:85–91CrossRef
34.
go back to reference Hu EL, Chen SC, Zhang DQ, Yin XS (2010) Semisupervised kernel matrix learning by kernel propagation. IEEE Trans Neural Netw 21(11):1831–1841CrossRef Hu EL, Chen SC, Zhang DQ, Yin XS (2010) Semisupervised kernel matrix learning by kernel propagation. IEEE Trans Neural Netw 21(11):1831–1841CrossRef
35.
go back to reference Zhang Y, Yeung DY (2011) Semisupervised generalized discriminant analysis. IEEE Trans Neural Netw 22(8):1207–1217CrossRef Zhang Y, Yeung DY (2011) Semisupervised generalized discriminant analysis. IEEE Trans Neural Netw 22(8):1207–1217CrossRef
36.
go back to reference Zhao M, Li B, Wu Z, Zhan C (2015) Image classification via least square semi-supervised discriminant analysis with flexible kernel regression for out-of-sample extension. Neurocomputing 153:96–107CrossRef Zhao M, Li B, Wu Z, Zhan C (2015) Image classification via least square semi-supervised discriminant analysis with flexible kernel regression for out-of-sample extension. Neurocomputing 153:96–107CrossRef
37.
go back to reference Faußer S, Schwenker F (2014) Semi-supervised clustering of large data sets with kernel methods. Pattern Recognit Lett 37:78–84CrossRef Faußer S, Schwenker F (2014) Semi-supervised clustering of large data sets with kernel methods. Pattern Recognit Lett 37:78–84CrossRef
38.
go back to reference Xiao JX, Hays J, Ehinger KA, Oliva A, Torralba A (2010) SUN database: large-scale scene recognition from abbey to zoo. In: IEEE conference on computer vision and pattern recognition, pp 3485–3492 Xiao JX, Hays J, Ehinger KA, Oliva A, Torralba A (2010) SUN database: large-scale scene recognition from abbey to zoo. In: IEEE conference on computer vision and pattern recognition, pp 3485–3492
39.
go back to reference Tipping ME (2000) Sparse kernel principal component analysis. Adv Neural Inf Process Syst 13:633–639 Tipping ME (2000) Sparse kernel principal component analysis. Adv Neural Inf Process Syst 13:633–639
40.
go back to reference Jiang XH, Snapp RR, Motai YC, Zhu XQ (2006) Accelerated kernel feature analysis. In: IEEE conference on computer vision and pattern recognition, pp 109–116 Jiang XH, Snapp RR, Motai YC, Zhu XQ (2006) Accelerated kernel feature analysis. In: IEEE conference on computer vision and pattern recognition, pp 109–116
41.
go back to reference Franc V, Hlavac V (2006) Greedy kernel principal component analysis. Lect Notes Comput Sci 3948:87–105CrossRef Franc V, Hlavac V (2006) Greedy kernel principal component analysis. Lect Notes Comput Sci 3948:87–105CrossRef
42.
go back to reference Xu Y, Zhang D, Jin Z, Li M, Yang JY (2006) A fast kernel-based nonlinear discriminant analysis for multi-class problems. Pattern Recognit 39(6):1026–1033CrossRefMATH Xu Y, Zhang D, Jin Z, Li M, Yang JY (2006) A fast kernel-based nonlinear discriminant analysis for multi-class problems. Pattern Recognit 39(6):1026–1033CrossRefMATH
43.
go back to reference Zeng WJ, Li XL, Zhang XD, Cheng E (2010) Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems. Signal Process 90(8):2333–2343CrossRefMATH Zeng WJ, Li XL, Zhang XD, Cheng E (2010) Kernel-based nonlinear discriminant analysis using minimum squared errors criterion for multiclass and undersampled problems. Signal Process 90(8):2333–2343CrossRefMATH
44.
go back to reference Cai D, He XF, Han JW (2011) Speed up kernel discriminant analysis. Int J Very Large Data Bases 20(1):21–33CrossRef Cai D, He XF, Han JW (2011) Speed up kernel discriminant analysis. Int J Very Large Data Bases 20(1):21–33CrossRef
45.
go back to reference Wang HX, Hu ZL, Zhao YE (2007) An efficient algorithm for generalized discriminant analysis using incomplete Cholesky decomposition. Pattern Recognit Lett 28(2):254–259CrossRef Wang HX, Hu ZL, Zhao YE (2007) An efficient algorithm for generalized discriminant analysis using incomplete Cholesky decomposition. Pattern Recognit Lett 28(2):254–259CrossRef
46.
go back to reference Li M, Bi W, Kwok JT, Lu BL (2015) Large-scale Nyström kernel matrix approximation using randomized SVD. IEEE Trans Neural Netw Learn Syst 26(1):152–164MathSciNetCrossRef Li M, Bi W, Kwok JT, Lu BL (2015) Large-scale Nyström kernel matrix approximation using randomized SVD. IEEE Trans Neural Netw Learn Syst 26(1):152–164MathSciNetCrossRef
47.
go back to reference Sun P, Yao X (2010) Sparse approximation through boosting for learning large scale kernel machines. IEEE Trans Neural Netw 21(6):883–894CrossRef Sun P, Yao X (2010) Sparse approximation through boosting for learning large scale kernel machines. IEEE Trans Neural Netw 21(6):883–894CrossRef
48.
go back to reference Rahimi A, Recht B (2009) Random features for large-scale kernel machines. Adv Neural Inf Process Syst: 1–10 Rahimi A, Recht B (2009) Random features for large-scale kernel machines. Adv Neural Inf Process Syst: 1–10
49.
go back to reference Zhou ZH, Chindaro S, Deravi F (2009) A classification framework for large-scale face recognition systems. Lect Notes Comput Sci 5558:337–346CrossRef Zhou ZH, Chindaro S, Deravi F (2009) A classification framework for large-scale face recognition systems. Lect Notes Comput Sci 5558:337–346CrossRef
50.
go back to reference Vedaldi A, Zisserman A (2012) Sparse kernel approximations for efficient classification and detection. In: IEEE conference on computer vision and pattern recognition, pp 2320–2327 Vedaldi A, Zisserman A (2012) Sparse kernel approximations for efficient classification and detection. In: IEEE conference on computer vision and pattern recognition, pp 2320–2327
51.
go back to reference Macua SV, Belanovic P, Zazo S (2011) Distributed linear discriminant analysis. In: International conference on acoustics, speech and signal processing, pp 3288–3291 Macua SV, Belanovic P, Zazo S (2011) Distributed linear discriminant analysis. In: International conference on acoustics, speech and signal processing, pp 3288–3291
52.
go back to reference Jing XY, Li S, Zhang D, Yang J, Yang JY (2012) Supervised and unsupervised parallel subspace learning for large-scale image recognition. IEEE Trans Circuits Syst Video Technol 22(10):1497–1511CrossRef Jing XY, Li S, Zhang D, Yang J, Yang JY (2012) Supervised and unsupervised parallel subspace learning for large-scale image recognition. IEEE Trans Circuits Syst Video Technol 22(10):1497–1511CrossRef
53.
go back to reference Fu JS, Yang WL (2011) Distributed kernel Fisher discriminant analysis for radar image recognition. In: International conference on mechanic automation and control engineering, pp 1241–1244 Fu JS, Yang WL (2011) Distributed kernel Fisher discriminant analysis for radar image recognition. In: International conference on mechanic automation and control engineering, pp 1241–1244
54.
go back to reference Ma ZY, Leijion A (2009) Bata mixture models and the application to image classification. In: International conference on image processing, pp 2045–2048 Ma ZY, Leijion A (2009) Bata mixture models and the application to image classification. In: International conference on image processing, pp 2045–2048
55.
go back to reference Mizukami Y, Tadamura K, Warrell J, Li P, Prince S (2010) CUDA implementation of deformable pattern recognition and its application to MNIST handwritten digit database. In: International conference on pattern recognition, pp 2001–2004 Mizukami Y, Tadamura K, Warrell J, Li P, Prince S (2010) CUDA implementation of deformable pattern recognition and its application to MNIST handwritten digit database. In: International conference on pattern recognition, pp 2001–2004
56.
go back to reference Lee KC, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698CrossRef Lee KC, Ho J, Kriegman D (2005) Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans Pattern Anal Mach Intell 27(5):684–698CrossRef
57.
go back to reference Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res 23:331–366MATH Chawla NV, Karakoulas GI (2005) Learning from labeled and unlabeled data: an empirical study across techniques and domains. J Artif Intell Res 23:331–366MATH
58.
go back to reference Turk MA, Pentland AP (1991) Face recognition using Eigenfaces. IEEE Conference on computer vision and pattern recognition, pp 586–591 Turk MA, Pentland AP (1991) Face recognition using Eigenfaces. IEEE Conference on computer vision and pattern recognition, pp 586–591
59.
go back to reference Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239CrossRef Kittler J, Hatef M, Duin RPW, Matas J (1998) On combining classifiers. IEEE Trans Pattern Anal Mach Intell 20(3):226–239CrossRef
Metadata
Title
Large-scale image recognition based on parallel kernel supervised and semi-supervised subspace learning
Authors
Fei Wu
Xiao-Yuan Jing
Qian Liu
Song-Song Wu
Guo-Liang He
Publication date
24-10-2015
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 3/2017
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-015-2081-y

Other articles of this Issue 3/2017

Neural Computing and Applications 3/2017 Go to the issue

Premium Partner