Skip to main content
main-content
Top

Hint

Swipe to navigate through the chapters of this book

2018 | OriginalPaper | Chapter

Robust PCAs and PCA Using Generalized Mean

Authors: Jiyong Oh, Nojun Kwak

Published in: Advances in Principal Component Analysis

Publisher: Springer Singapore

share
SHARE

Abstract

In this chapter, a robust principal component analysis (PCA) is described, which can overcome the problem that PCA is prone to outliers included in training set. Different from the other alternatives which commonly replace \(L_{2}\)-norm by other distance measures, our method alleviates the negative effect of outliers using the characteristic of the generalized mean keeping the use of the Euclidean distance. The optimization problem based on the generalized mean is solved by a novel method. We also present a generalized sample mean, which is a generalization of the sample mean, to estimate a robust mean in the presence of outliers. The proposed method shows better or equivalent performance than the conventional PCAs in various problems such as face reconstruction, clustering, and object categorization.

To get access to this content you need the following product:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 69.000 Bücher
  • über 500 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 50.000 Bücher
  • über 380 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




Testen Sie jetzt 15 Tage kostenlos.

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 58.000 Bücher
  • über 300 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Testen Sie jetzt 15 Tage kostenlos.

Appendix
Available only for authorised users
Literature
1.
go back to reference Jain, A., Duin, R., Jianchang, M.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Machine Intell. 22(1), 4–37 (2000) CrossRef Jain, A., Duin, R., Jianchang, M.: Statistical pattern recognition: a review. IEEE Trans. Pattern Anal. Machine Intell. 22(1), 4–37 (2000) CrossRef
2.
go back to reference Jolliffe, I.: Principal Component Analysis, 2nd edn. Springer, Berlin (2002) MATH Jolliffe, I.: Principal Component Analysis, 2nd edn. Springer, Berlin (2002) MATH
3.
go back to reference Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Machine Intell. 19(7), 711–720 (1997) CrossRef Belhumeur, P., Hespanha, J., Kriegman, D.: Eigenfaces versus fisherfaces: recognition using class specific linear projection. IEEE Trans. Pattern Anal. Machine Intell. 19(7), 711–720 (1997) CrossRef
4.
go back to reference Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neurosci. 3(1), 71–86 (1991) CrossRef Turk, M., Pentland, A.: Eigenfaces for recognition. J. Cognitive Neurosci. 3(1), 71–86 (1991) CrossRef
5.
go back to reference Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vision 77(1–3), 125–141 (2008) CrossRef Ross, D.A., Lim, J., Lin, R.S., Yang, M.H.: Incremental learning for robust visual tracking. Int. J. Comput. Vision 77(1–3), 125–141 (2008) CrossRef
6.
go back to reference Ding, C., He, X.: \(K\)-means Clustering via Principal Component Analysis. In: Proceedings of the 21st International Conference on Machine Learning, ICML ’04 (2004) Ding, C., He, X.: \(K\)-means Clustering via Principal Component Analysis. In: Proceedings of the 21st International Conference on Machine Learning, ICML ’04 (2004)
7.
go back to reference Yeung, K.Y., Ruzzo, W.L.: Principal component analysis for clustering gene expression data. Bioinformatics 17(9), 763–774 (2001) CrossRef Yeung, K.Y., Ruzzo, W.L.: Principal component analysis for clustering gene expression data. Bioinformatics 17(9), 763–774 (2001) CrossRef
8.
go back to reference de la Torre, F., Black, M.J.: A framework for robust subspace learning. Int. J. Comput. Vision 54(1–3), 117–142 (2003) CrossRefMATH de la Torre, F., Black, M.J.: A framework for robust subspace learning. Int. J. Comput. Vision 54(1–3), 117–142 (2003) CrossRefMATH
9.
go back to reference Brooks, J., Dulá, J., Boone, E.: A pure \(L_{1}\)-norm principal component analysis. Comput. Statist. Data Anal. 61, 83–98 (2013) MathSciNetCrossRefMATH Brooks, J., Dulá, J., Boone, E.: A pure \(L_{1}\)-norm principal component analysis. Comput. Statist. Data Anal. 61, 83–98 (2013) MathSciNetCrossRefMATH
10.
go back to reference Ding, C., Zhou, D., He, X., Zha, H.: \(R_{1}\)-PCA: Rotational Invariant \(L_{1}\)-norm Principal Component Analysis for Robust Subspace Factorization. In: Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, pp. 281–288 (2006) Ding, C., Zhou, D., He, X., Zha, H.: \(R_{1}\)-PCA: Rotational Invariant \(L_{1}\)-norm Principal Component Analysis for Robust Subspace Factorization. In: Proceedings of the 23rd International Conference on Machine Learning, ICML ’06, pp. 281–288 (2006)
11.
go back to reference He, R., Hu, B., Yuan, X., Zheng, W.S.: Principal component analysis based on non-parametric maximum entropy. Neurocomputing 73(10–12), 1840–1852 (2010) CrossRef He, R., Hu, B., Yuan, X., Zheng, W.S.: Principal component analysis based on non-parametric maximum entropy. Neurocomputing 73(10–12), 1840–1852 (2010) CrossRef
12.
go back to reference He, R., Hu, B.G., Zheng, W.S., Kong, X.W.: Robust principal component analysis based on maximum correntropy criterion. IEEE Trans. Image Process. 20(6), 1485–1494 (2011) MathSciNetCrossRefMATH He, R., Hu, B.G., Zheng, W.S., Kong, X.W.: Robust principal component analysis based on maximum correntropy criterion. IEEE Trans. Image Process. 20(6), 1485–1494 (2011) MathSciNetCrossRefMATH
13.
go back to reference Ke, Q., Kanade, T.: Robust \(L_{1}\) Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, vol. 1, pp. 739–746 (2005) Ke, Q., Kanade, T.: Robust \(L_{1}\) Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2005, vol. 1, pp. 739–746 (2005)
14.
go back to reference Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Machine Intell. 30(9), 1672–1680 (2008) CrossRef Kwak, N.: Principal component analysis based on L1-norm maximization. IEEE Trans. Pattern Anal. Machine Intell. 30(9), 1672–1680 (2008) CrossRef
15.
go back to reference Kwak, N.: Principal component analysis by \(l_{p}\)-norm maximization. IEEE Trans. Cyber. 44(5), 594–609 (2014) CrossRef Kwak, N.: Principal component analysis by \(l_{p}\)-norm maximization. IEEE Trans. Cyber. 44(5), 594–609 (2014) CrossRef
16.
go back to reference Liang, Z., Xia, S., Zhou, Y., Zhang, L., Li, Y.: Feature extraction based on \(L_{p}\)-norm generalized principal component analysis. Pattern Recognit. Lett. 34(9), 1037–1045 (2013) CrossRef Liang, Z., Xia, S., Zhou, Y., Zhang, L., Li, Y.: Feature extraction based on \(L_{p}\)-norm generalized principal component analysis. Pattern Recognit. Lett. 34(9), 1037–1045 (2013) CrossRef
17.
go back to reference Ng, A.Y.: Feature selection, \(L_{1}\) vs. \(L_{2}\) regularization, and rotational invariance. In: Proceedings of the 21st International Conference on Machine Learning, ICML ’04 (2004) Ng, A.Y.: Feature selection, \(L_{1}\) vs. \(L_{2}\) regularization, and rotational invariance. In: Proceedings of the 21st International Conference on Machine Learning, ICML ’04 (2004)
18.
go back to reference Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1981) MATH Cover, T.M., Thomas, J.A.: Elements of Information Theory. Wiley, New York (1981) MATH
19.
go back to reference Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans. Signal Process. 55(11), 5286–5298 (2007) MathSciNetCrossRef Liu, W., Pokharel, P.P., Principe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans. Signal Process. 55(11), 5286–5298 (2007) MathSciNetCrossRef
20.
go back to reference Bullen, P.: Handbook of Means and Their Inequalities, 2nd edn. Kluwer Academic Publisher, Dordrecht (2003) CrossRefMATH Bullen, P.: Handbook of Means and Their Inequalities, 2nd edn. Kluwer Academic Publisher, Dordrecht (2003) CrossRefMATH
21.
go back to reference Oh, J., Kwak, N.: Generalized mean for robust principal component analysis. Pattern Recognit. 54, 116–127 (2016) CrossRef Oh, J., Kwak, N.: Generalized mean for robust principal component analysis. Pattern Recognit. 54, 116–127 (2016) CrossRef
22.
go back to reference Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust Principal Component Analysis? J. ACM 58(3), 11:1–11:37 (2011) Candès, E.J., Li, X., Ma, Y., Wright, J.: Robust Principal Component Analysis? J. ACM 58(3), 11:1–11:37 (2011)
24.
go back to reference Golub, G.H., Loan, C.F.V.: Matrix Computations, 3rd edn. Johns Hopkins, Baltimore (1996) MATH Golub, G.H., Loan, C.F.V.: Matrix Computations, 3rd edn. Johns Hopkins, Baltimore (1996) MATH
25.
go back to reference Oh, J., Kwak, N., Lee, M., Choi, C.H.: Generalized mean for feature extraction in one-class classification problems. Pattern Recognit. 46(12), 3328–3340 (2013) CrossRef Oh, J., Kwak, N., Lee, M., Choi, C.H.: Generalized mean for feature extraction in one-class classification problems. Pattern Recognit. 46(12), 3328–3340 (2013) CrossRef
26.
go back to reference Bishop, C.M.: Pattern Recongntion and Machine Learning. Springer, Berlin (2006) Bishop, C.M.: Pattern Recongntion and Machine Learning. Springer, Berlin (2006)
27.
go back to reference Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine Intell. 24(5), 603–619 (2002) CrossRef Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Machine Intell. 24(5), 603–619 (2002) CrossRef
28.
go back to reference Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Machine Intell. 22(10), 1090–1104 (2000) CrossRef Phillips, P., Moon, H., Rizvi, S., Rauss, P.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Machine Intell. 22(10), 1090–1104 (2000) CrossRef
29.
go back to reference LeCun, Y., Huang, F.J., Bottou, L.: Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II–97–104 (2004) LeCun, Y., Huang, F.J., Bottou, L.: Learning Methods for Generic Object Recognition with Invariance to Pose and Lighting. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 2, pp. II–97–104 (2004)
30.
go back to reference Hauberg, S., Feragen, A., Black, M.: Grassmann Averages for Scalable Robust PCA. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3810–3817 (2014) Hauberg, S., Feragen, A., Black, M.: Grassmann Averages for Scalable Robust PCA. In: Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3810–3817 (2014)
Metadata
Title
Robust PCAs and PCA Using Generalized Mean
Authors
Jiyong Oh
Nojun Kwak
Copyright Year
2018
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-6704-4_4