Skip to main content
Erschienen in: Pattern Analysis and Applications 3/2018

07.02.2017 | Theoretical Advances

A new linear discriminant analysis algorithm based on L1-norm maximization and locality preserving projection

verfasst von: Di Zhang, Xueqiang Li, Jiazhong He, Minghui Du

Erschienen in: Pattern Analysis and Applications | Ausgabe 3/2018

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Generic L2-norm-based linear discriminant analysis (LDA) is sensitive to outliers and only captures global structure information of sample points. In this paper, a new LDA-based feature extraction algorithm is proposed to integrate both global and local structure information via a unified L1-norm optimization framework. Unlike generic L2-norm-based LDA, the proposed algorithm explicitly incorporates the local structure information of sample points and is robust to outliers. It overcomes the problem of the singularity of within-class scatter matrix as well. Experiments on several popular datasets demonstrate the effectiveness of the proposed algorithm.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Belhumeur PN, Hepanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef Belhumeur PN, Hepanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE Trans Pattern Anal Mach Intell 19(7):711–720CrossRef
2.
Zurück zum Zitat Martinez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233CrossRef Martinez AM, Kak AC (2001) PCA versus LDA. IEEE Trans Pattern Anal Mach Intell 23(2):228–233CrossRef
3.
Zurück zum Zitat Ji S-W, Ye J-P (2008) Generalized linear discriminant analysis: a unified frame-work and efficient model selection. IEEE Trans Neural Netw 19(10):1768–1782CrossRef Ji S-W, Ye J-P (2008) Generalized linear discriminant analysis: a unified frame-work and efficient model selection. IEEE Trans Neural Netw 19(10):1768–1782CrossRef
4.
Zurück zum Zitat Lu J-W, Plataniotis K, Venetsanopoulos A (2005) Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recogn Lett 26:181–191CrossRef Lu J-W, Plataniotis K, Venetsanopoulos A (2005) Regularization studies of linear discriminant analysis in small sample size scenarios with application to face recognition. Pattern Recogn Lett 26:181–191CrossRef
5.
Zurück zum Zitat Ye J, Li Q (2005) A two-stage linear discriminant analysis via QR decomposition. IEEE Trans Pattern Anal Mach Intell 27(6):929–941CrossRef Ye J, Li Q (2005) A two-stage linear discriminant analysis via QR decomposition. IEEE Trans Pattern Anal Mach Intell 27(6):929–941CrossRef
6.
Zurück zum Zitat Howland P, Park H (2004) Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Trans Pattern Anal Mach Intell 26(8):995–1006CrossRef Howland P, Park H (2004) Generalizing discriminant analysis using the generalized singular value decomposition. IEEE Trans Pattern Anal Mach Intell 26(8):995–1006CrossRef
7.
Zurück zum Zitat Ye J-P (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204MathSciNetMATH Ye J-P (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204MathSciNetMATH
8.
Zurück zum Zitat Chen L-F, Mark Liao HY, Ko MT, Lin J-C, Yu G-J (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn 33:1713–1726CrossRef Chen L-F, Mark Liao HY, Ko MT, Lin J-C, Yu G-J (2000) A new LDA-based face recognition system which can solve the small sample size problem. Pattern Recogn 33:1713–1726CrossRef
9.
Zurück zum Zitat Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data-with application to face recognition. Pattern Recogn 34:2067–2070MATHCrossRef Yu H, Yang J (2001) A direct LDA algorithm for high-dimensional data-with application to face recognition. Pattern Recogn 34:2067–2070MATHCrossRef
10.
Zurück zum Zitat Cai D, He X, Han J (2008) SRDA: an efficient algorithm for large-scale discriminant analysis. IEEE Trans Knowl Data Eng 20(4):1–12CrossRef Cai D, He X, Han J (2008) SRDA: an efficient algorithm for large-scale discriminant analysis. IEEE Trans Knowl Data Eng 20(4):1–12CrossRef
11.
Zurück zum Zitat Yang J, Yang J-Y (2003) Why can LDA be performed in PCA transformed space? Pattern Recogn 36:563–566CrossRef Yang J, Yang J-Y (2003) Why can LDA be performed in PCA transformed space? Pattern Recogn 36:563–566CrossRef
12.
Zurück zum Zitat Hou C, Zhang C, Wu Y, Jiao Y (2009) Stable local dimensionality reduction approaches. Pattern Recogn 42(9):2054–2066MATHCrossRef Hou C, Zhang C, Wu Y, Jiao Y (2009) Stable local dimensionality reduction approaches. Pattern Recogn 42(9):2054–2066MATHCrossRef
13.
Zurück zum Zitat Gao Q, Xu H, Li Y, Xie D (2010) Two-dimensional supervised local similarity and diversity projection. Pattern Recogn 43(10):3359–3363MATHCrossRef Gao Q, Xu H, Li Y, Xie D (2010) Two-dimensional supervised local similarity and diversity projection. Pattern Recogn 43(10):3359–3363MATHCrossRef
14.
Zurück zum Zitat Gao Q, Liu J, Zhang H, Hou J, Yang X (2012) Enhanced fisher discriminant criterion for image recognition. Pattern Recogn 45(10):3717–3724CrossRef Gao Q, Liu J, Zhang H, Hou J, Yang X (2012) Enhanced fisher discriminant criterion for image recognition. Pattern Recogn 45(10):3717–3724CrossRef
15.
Zurück zum Zitat Zhang D, He J, Zhao Y, Luo Z, Minghui D (2014) Global plus local: a complete framework for feature extraction and recognition. Pattern Recogn 47(3):1433–1442MATHCrossRef Zhang D, He J, Zhao Y, Luo Z, Minghui D (2014) Global plus local: a complete framework for feature extraction and recognition. Pattern Recogn 47(3):1433–1442MATHCrossRef
16.
Zurück zum Zitat Black MJ, Rangarajan A (1996) On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int J Comput Vis 19(1):57–91CrossRef Black MJ, Rangarajan A (1996) On the unification of line processes, outlier rejection, and robust statistics with applications in early vision. Int J Comput Vis 19(1):57–91CrossRef
17.
Zurück zum Zitat Pang Y, Li X, Yuan Y (2010) Robust tensor analysis with L1-norm. IEEE Trans Circuits Syst Video Technol 20(2):172–178CrossRef Pang Y, Li X, Yuan Y (2010) Robust tensor analysis with L1-norm. IEEE Trans Circuits Syst Video Technol 20(2):172–178CrossRef
18.
Zurück zum Zitat Wang H, Tang Q, Zheng W (2012) L1-norm-based common spatial patterns. IEEE Trans Biomed Eng 59(3):653–662CrossRef Wang H, Tang Q, Zheng W (2012) L1-norm-based common spatial patterns. IEEE Trans Biomed Eng 59(3):653–662CrossRef
19.
Zurück zum Zitat Ke Q, Kanade T (2003) Robust subspace computation using L1 norm. Carnegie Mellon University, Pittsburgh, Technical Report, CMU-CS-03-172 Ke Q, Kanade T (2003) Robust subspace computation using L1 norm. Carnegie Mellon University, Pittsburgh, Technical Report, CMU-CS-03-172
20.
Zurück zum Zitat Ke Q, Kanade T (2005) Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: Proceedings of the IEEE international conferences computer vision pattern recognition, pp 739–746 Ke Q, Kanade T (2005) Robust L1 norm factorization in the presence of outliers and missing data by alternative convex programming. In: Proceedings of the IEEE international conferences computer vision pattern recognition, pp 739–746
21.
Zurück zum Zitat Ding C, Zhou D, He X, Zha H (2006) R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings international conference on machine learning, pp 281–288 Ding C, Zhou D, He X, Zha H (2006) R1-PCA: rotational invariant L1-norm principal component analysis for robust subspace factorization. In: Proceedings international conference on machine learning, pp 281–288
22.
Zurück zum Zitat Kwak N (2008) Principal component analysis based on L1-norm maximization. IEEE Trans Pattern Anal Mach Intell 30(9):672–1680CrossRef Kwak N (2008) Principal component analysis based on L1-norm maximization. IEEE Trans Pattern Anal Mach Intell 30(9):672–1680CrossRef
23.
Zurück zum Zitat Li XL, Pang YW, Yuan Y (2009) L1-norm-based 2DPCA. IEEE Trans Syst Man Cybern B Cybern 40(4):1170–1175 Li XL, Pang YW, Yuan Y (2009) L1-norm-based 2DPCA. IEEE Trans Syst Man Cybern B Cybern 40(4):1170–1175
24.
Zurück zum Zitat Li X, Hua W, Wang H, Zhang Z (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579CrossRef Li X, Hua W, Wang H, Zhang Z (2010) Linear discriminant analysis using rotational invariant L1 norm. Neurocomputing 73(13–15):2571–2579CrossRef
25.
Zurück zum Zitat Zhou F, Zhang J (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027MathSciNetMATHCrossRef Zhou F, Zhang J (2013) Linear discriminant analysis based on L1-norm maximization. IEEE Trans Image Process 22(8):3018–3027MathSciNetMATHCrossRef
26.
Zurück zum Zitat Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRef Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRef
27.
Zurück zum Zitat Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323CrossRef Tenenbaum JB, de Silva V, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290(5500):2319–2323CrossRef
28.
Zurück zum Zitat Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 1:585–592 Belkin M, Niyogi P (2002) Laplacian eigenmaps and spectral techniques for embedding and clustering. Adv Neural Inf Process Syst 1:585–592
29.
Zurück zum Zitat Yan S, Xu D, Zhang B, Zhang H, Yang Q, Lin S (2007) “Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51CrossRef Yan S, Xu D, Zhang B, Zhang H, Yang Q, Lin S (2007) “Graph embedding and extensions: a general framework for dimensionality reduction. IEEE Trans Pattern Anal Mach Intell 29(1):40–51CrossRef
30.
Zurück zum Zitat Cai H-P, Mikolajczyk K, Matas J (2011) Learning linear discriminant projections for dimensionality reduction of image descriptors. IEEE Trans Pattern Anal Mach Intell 33(2):338–352CrossRef Cai H-P, Mikolajczyk K, Matas J (2011) Learning linear discriminant projections for dimensionality reduction of image descriptors. IEEE Trans Pattern Anal Mach Intell 33(2):338–352CrossRef
31.
Zurück zum Zitat Cui Y, Fan L (2012) A novel supervised dimensionality reduction algorithm: graph-based Fisher analysis. Pattern Recogn 45(4):1471–1481MATHCrossRef Cui Y, Fan L (2012) A novel supervised dimensionality reduction algorithm: graph-based Fisher analysis. Pattern Recogn 45(4):1471–1481MATHCrossRef
32.
Zurück zum Zitat He X, Niyogi P (2004) Locality preserving projections. Adv Neural Inf Process Syst 16:153–160 He X, Niyogi P (2004) Locality preserving projections. Adv Neural Inf Process Syst 16:153–160
33.
Zurück zum Zitat Shu X, Gao Y, Lu H (2012) Efficient linear discriminant analysis with locality preserving for face recognition. Pattern Recogn 45(5):1892–1898MATHCrossRef Shu X, Gao Y, Lu H (2012) Efficient linear discriminant analysis with locality preserving for face recognition. Pattern Recogn 45(5):1892–1898MATHCrossRef
34.
Zurück zum Zitat Masashi S (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8:1027–1061MATH Masashi S (2007) Dimensionality reduction of multimodal labeled data by local fisher discriminant analysis. J Mach Learn Res 8:1027–1061MATH
35.
Zurück zum Zitat Wang N, Tao D, Gao X, Liu W, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24(9):1364–1376CrossRef Wang N, Tao D, Gao X, Liu W, Li J (2013) Transductive face sketch-photo synthesis. IEEE Trans Neural Netw Learn Syst 24(9):1364–1376CrossRef
36.
Zurück zum Zitat Yang J, Zhang D, Yang J-Y, Niu B (2007) Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics. IEEE Trans Pattern Anal Mach Intell 29(4):650–664CrossRef Yang J, Zhang D, Yang J-Y, Niu B (2007) Globally maximizing, locally minimizing: unsupervised discriminant projection with applications to face and palm biometrics. IEEE Trans Pattern Anal Mach Intell 29(4):650–664CrossRef
37.
Zurück zum Zitat Loog M, Duin RPW (2004) Heteroscedastic extension of LDA: the Chernoff criterion. IEEE Trans Pattern Anal Mach Intell 26(6):732–739CrossRef Loog M, Duin RPW (2004) Heteroscedastic extension of LDA: the Chernoff criterion. IEEE Trans Pattern Anal Mach Intell 26(6):732–739CrossRef
38.
Zurück zum Zitat Li X, Hao Z, Huang H (2010) An evolution algorithm with sorted race mechanism for global optimization. In: Proceedings of international conference on machine learning and computing (ICMLC 2010), vol 3, pp 1550–1555 Li X, Hao Z, Huang H (2010) An evolution algorithm with sorted race mechanism for global optimization. In: Proceedings of international conference on machine learning and computing (ICMLC 2010), vol 3, pp 1550–1555
40.
Zurück zum Zitat Yang J, Frangi AF, Yang J, Zhang D, Jin Z (2005) KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans Pattern Anal Mach Intell 27(2):230–244CrossRef Yang J, Frangi AF, Yang J, Zhang D, Jin Z (2005) KPCA plus LDA: a complete kernel Fisher discriminant framework for feature extraction and recognition. IEEE Trans Pattern Anal Mach Intell 27(2):230–244CrossRef
41.
Zurück zum Zitat Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans Neural Netw 14(1):117–126CrossRef Lu J, Plataniotis KN, Venetsanopoulos AN (2003) Face recognition using kernel direct discriminant analysis algorithms. IEEE Trans Neural Netw 14(1):117–126CrossRef
44.
Zurück zum Zitat Tsai JT, Liu TK, Chou JH (2004) Hybrid Taguchi-genetic algorithm for global numerical optimization. IEEE Trans Evol Comput 8(4):365–377CrossRef Tsai JT, Liu TK, Chou JH (2004) Hybrid Taguchi-genetic algorithm for global numerical optimization. IEEE Trans Evol Comput 8(4):365–377CrossRef
45.
Zurück zum Zitat Leung YW, Wang YP (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Trans Evol Comput 5(1):41–53CrossRef Leung YW, Wang YP (2001) An orthogonal genetic algorithm with quantization for global numerical optimization. IEEE Trans Evol Comput 5(1):41–53CrossRef
46.
Zurück zum Zitat Jenatton R, Obozinski G, Bach F (2010) Structured sparse principal component analysis. In: Proceedings of the international conference artificial intelligence statistics, pp 1–8 Jenatton R, Obozinski G, Bach F (2010) Structured sparse principal component analysis. In: Proceedings of the international conference artificial intelligence statistics, pp 1–8
47.
Zurück zum Zitat Zhong F, Zhang J, Li D (2014) Discriminant locality preserving projections based on L1-norm maximization. IEEE Trans Neural Netw Learn Syst 25(11):2065–2074CrossRef Zhong F, Zhang J, Li D (2014) Discriminant locality preserving projections based on L1-norm maximization. IEEE Trans Neural Netw Learn Syst 25(11):2065–2074CrossRef
Metadaten
Titel
A new linear discriminant analysis algorithm based on L1-norm maximization and locality preserving projection
verfasst von
Di Zhang
Xueqiang Li
Jiazhong He
Minghui Du
Publikationsdatum
07.02.2017
Verlag
Springer London
Erschienen in
Pattern Analysis and Applications / Ausgabe 3/2018
Print ISSN: 1433-7541
Elektronische ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-017-0594-y

Weitere Artikel der Ausgabe 3/2018

Pattern Analysis and Applications 3/2018 Zur Ausgabe