Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 6/2013

01.12.2013 | Original Article

Principal component analysis using QR decomposition

verfasst von: Alok Sharma, Kuldip K. Paliwal, Seiya Imoto, Satoru Miyano

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 6/2013

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper we present QR based principal component analysis (PCA) method. Similar to the singular value decomposition (SVD) based PCA method this method is numerically stable. We have carried out analytical comparison as well as numerical comparison (on Matlab software) to investigate the performance (in terms of computational complexity) of our method. The computational complexity of SVD based PCA is around \( 14dn^{2} \) flops (where d is the dimensionality of feature space and n is the number of training feature vectors); whereas the computational complexity of QR based PCA is around \( 2dn^{2} \, + \,2dth \) flops (where t is the rank of data covariance matrix and h is the dimensionality of reduced feature space). It is observed that the QR based PCA is more efficient in terms of computational complexity.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Fußnoten
1
To deal with higher order matrices, other variants of PCA have been proposed in the literature. See the following references for details [2327]
 
Literatur
1.
Zurück zum Zitat Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part I. Principal component analysis. Chemom Intell Lab Syst 24:31–42CrossRef Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part I. Principal component analysis. Chemom Intell Lab Syst 24:31–42CrossRef
2.
Zurück zum Zitat Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE T PAMI 19(7):711–720CrossRef Belhumeur PN, Hespanha JP, Kriegman DJ (1997) Eigenfaces vs. fisherfaces: recognition using class specific linear projection. IEEE T PAMI 19(7):711–720CrossRef
3.
Zurück zum Zitat Chen T-C, Chen K, Liu W, Chen L-G (2009) One-chip principal component analysis with a mean pre-estimation method for spike sorting. In: Proceedings of IEEE international symposium on circuits and systems, pp 3310–3113 Chen T-C, Chen K, Liu W, Chen L-G (2009) One-chip principal component analysis with a mean pre-estimation method for spike sorting. In: Proceedings of IEEE international symposium on circuits and systems, pp 3310–3113
4.
Zurück zum Zitat Chen T-C, Liu W, Chen L-G (2008) VLSI architecture of leading eigenvector generation for on-chip principal component analysis spike sorting system. In: Proceedings of 30th Annual International Conference IEEE engineering in medicine and biology society EMBS’08, pp 3192–3195 Chen T-C, Liu W, Chen L-G (2008) VLSI architecture of leading eigenvector generation for on-chip principal component analysis spike sorting system. In: Proceedings of 30th Annual International Conference IEEE engineering in medicine and biology society EMBS’08, pp 3192–3195
5.
Zurück zum Zitat Fukunaga K (1990) Introduction to Statistical Pattern Recognition. Academic Press, USAMATH Fukunaga K (1990) Introduction to Statistical Pattern Recognition. Academic Press, USAMATH
6.
Zurück zum Zitat Golub GH, Loan CFV (1996) Matrix Computations. The John Hopkins University Press, BaltimoreMATH Golub GH, Loan CFV (1996) Matrix Computations. The John Hopkins University Press, BaltimoreMATH
7.
Zurück zum Zitat Jian H, Yuen PC, Wen-Sheng C (2004) A novel subspace LDA algorithms for recognition of face images with illumination and pose variations. Proc Int Conf Mach Learn Cybern 6:3589–3594 Jian H, Yuen PC, Wen-Sheng C (2004) A novel subspace LDA algorithms for recognition of face images with illumination and pose variations. Proc Int Conf Mach Learn Cybern 6:3589–3594
8.
Zurück zum Zitat Kiers HAL, Harshman RA (1997) Relating two proposed methods for speed up of algorithms for fitting two- and three- way principal component and related multilinear models. Chemom Intell Lab Syst 36:31–40CrossRef Kiers HAL, Harshman RA (1997) Relating two proposed methods for speed up of algorithms for fitting two- and three- way principal component and related multilinear models. Chemom Intell Lab Syst 36:31–40CrossRef
9.
Zurück zum Zitat Paliwal KK, Sharma A (2010) Improved direct LDA and its application to DNA microarray gene expression data. Pattern Recogn Lett 31:2489–2492CrossRef Paliwal KK, Sharma A (2010) Improved direct LDA and its application to DNA microarray gene expression data. Pattern Recogn Lett 31:2489–2492CrossRef
10.
Zurück zum Zitat Roweis S (1997) EM Algorithms for PCA and SPCA. Neural Inf Process Syst 10:626–632 Roweis S (1997) EM Algorithms for PCA and SPCA. Neural Inf Process Syst 10:626–632
11.
Zurück zum Zitat Sajid I, Ahmed MM, Taj I (2008) Design and implementation of a face recognition system using fast PCA. In: Proceedings of the International Symposium on Computer Science and its Applications CSA, pp 126–130 Sajid I, Ahmed MM, Taj I (2008) Design and implementation of a face recognition system using fast PCA. In: Proceedings of the International Symposium on Computer Science and its Applications CSA, pp 126–130
12.
Zurück zum Zitat Schilling RJ, Harris SL (2000) Applied Numerical Methods for Engineers Using Matlab and C. Brooks/Cole Publishing, Boston Schilling RJ, Harris SL (2000) Applied Numerical Methods for Engineers Using Matlab and C. Brooks/Cole Publishing, Boston
13.
Zurück zum Zitat Sharma A, Paliwal KK (2010) Regularisation of eigenfeatures by extrapolation of scatter-matrix in face-recognition problem. Electron Lett IEE 46(10):682–683CrossRef Sharma A, Paliwal KK (2010) Regularisation of eigenfeatures by extrapolation of scatter-matrix in face-recognition problem. Electron Lett IEE 46(10):682–683CrossRef
14.
Zurück zum Zitat Sharma A, Paliwal KK (2007) Fast principal component analysis using fixed-point algorithm. Pattern Recogn Lett 28(10):1151–1155CrossRef Sharma A, Paliwal KK (2007) Fast principal component analysis using fixed-point algorithm. Pattern Recogn Lett 28(10):1151–1155CrossRef
15.
Zurück zum Zitat Sirovich L (1987) Turbulence and the dynamics of coherent structures. Quart Appl Math XLV 45:561–571MathSciNetMATH Sirovich L (1987) Turbulence and the dynamics of coherent structures. Quart Appl Math XLV 45:561–571MathSciNetMATH
16.
Zurück zum Zitat Song F, Zhang D, Wang J, Liu H, Tao Q (2007) A parameterized direct LDA and its application to face recognition. Neurocomputing 71:191–196CrossRef Song F, Zhang D, Wang J, Liu H, Tao Q (2007) A parameterized direct LDA and its application to face recognition. Neurocomputing 71:191–196CrossRef
17.
Zurück zum Zitat Swets DL, Weng J (1996) Using discriminative eigenfeatures for image retrieval. IEEE T PAMI 18(8):831–836CrossRef Swets DL, Weng J (1996) Using discriminative eigenfeatures for image retrieval. IEEE T PAMI 18(8):831–836CrossRef
18.
Zurück zum Zitat Wu X-J, Kittler J, Yang J-Y, Messerm K, Wang S (2004) A new direct LDA (D-LDA) algorithm for feature extraction in face recognition. Proc Int Conf Pattern Recognit 4:545–548 Wu X-J, Kittler J, Yang J-Y, Messerm K, Wang S (2004) A new direct LDA (D-LDA) algorithm for feature extraction in face recognition. Proc Int Conf Pattern Recognit 4:545–548
19.
Zurück zum Zitat Xiaogang W, Xiaoou T (2004) A unified framework for subspace face recognition. IEEE T PAMI 26(9):1222–1228CrossRef Xiaogang W, Xiaoou T (2004) A unified framework for subspace face recognition. IEEE T PAMI 26(9):1222–1228CrossRef
20.
Zurück zum Zitat Ye J, Xiong T (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204MathSciNetMATH Ye J, Xiong T (2006) Computational and theoretical analysis of null space and orthogonal linear discriminant analysis. J Mach Learn Res 7:1183–1204MathSciNetMATH
21.
Zurück zum Zitat Zhao W, Chellappa R, Nandhakumar N (1998) Empirical performance analysis of linear discriminant classifiers. In: Proceedings of IEEE Conferences on Computer Vision and Pattern Recognition, pp 164–169 Zhao W, Chellappa R, Nandhakumar N (1998) Empirical performance analysis of linear discriminant classifiers. In: Proceedings of IEEE Conferences on Computer Vision and Pattern Recognition, pp 164–169
22.
Zurück zum Zitat Sharma A, Paliwal KK (2008) A Gradient linear discriminant analysis for small sample sized problem. Neural Process Lett 27(1):17–24CrossRef Sharma A, Paliwal KK (2008) A Gradient linear discriminant analysis for small sample sized problem. Neural Process Lett 27(1):17–24CrossRef
23.
Zurück zum Zitat Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part II. Three-mode principal component analysis. Chemom Intell Lab Syst 24:43–54CrossRef Alsberg BK, Kvalheim OM (1994) Speed improvement of multivariate algorithms by the method of postponed basis matrix multiplication Part II. Three-mode principal component analysis. Chemom Intell Lab Syst 24:43–54CrossRef
24.
Zurück zum Zitat Carroll JD, Pruzansky S, Kruskal JB (1980) CANELINC: a general approach to multidimensional analysis of many-way arrays with linear constraints on parameters. Psychometrika 45:3–24MathSciNetCrossRefMATH Carroll JD, Pruzansky S, Kruskal JB (1980) CANELINC: a general approach to multidimensional analysis of many-way arrays with linear constraints on parameters. Psychometrika 45:3–24MathSciNetCrossRefMATH
25.
Zurück zum Zitat Giordani P, Kiers HAL (2004) Three-way component analysis of interval-valued data. J Chemom 18:253–264CrossRef Giordani P, Kiers HAL (2004) Three-way component analysis of interval-valued data. J Chemom 18:253–264CrossRef
26.
Zurück zum Zitat Kiers HAL, Harshman RA (2009) An efficient algorithm for Parafac with uncorrelated mode-A components applied to large I × J × K datasets with JK. J Chemom 23:442–447CrossRef Kiers HAL, Harshman RA (2009) An efficient algorithm for Parafac with uncorrelated mode-A components applied to large I × J × K datasets with JK. J Chemom 23:442–447CrossRef
27.
Zurück zum Zitat Kiers HAL, Kroonenberg PM, Berge JMFT (1992) An efficient algorithm for Tuckals3 on data with large numbers of observation units. Psychometrika 57:415–422CrossRef Kiers HAL, Kroonenberg PM, Berge JMFT (1992) An efficient algorithm for Tuckals3 on data with large numbers of observation units. Psychometrika 57:415–422CrossRef
28.
Zurück zum Zitat Poon B, Amin MA, Yan H (2011) Performance evaluation and comparison of PCA Based human face recognition methods for distorted images. Int J Mach Learn Cybern 2(4):245–259CrossRef Poon B, Amin MA, Yan H (2011) Performance evaluation and comparison of PCA Based human face recognition methods for distorted images. Int J Mach Learn Cybern 2(4):245–259CrossRef
29.
Zurück zum Zitat Sharma A, Paliwal KK (2012) A two-stage linear discriminant analysis for face-recognition. Pattern Recogn Lett 33(9):1157–1162CrossRef Sharma A, Paliwal KK (2012) A two-stage linear discriminant analysis for face-recognition. Pattern Recogn Lett 33(9):1157–1162CrossRef
30.
Zurück zum Zitat Pei D, Mi J-S (2011) Attribute reduction in decision formal context based on homomorphism. Int J Mach Learn Cybern 2(4):289–293CrossRef Pei D, Mi J-S (2011) Attribute reduction in decision formal context based on homomorphism. Int J Mach Learn Cybern 2(4):289–293CrossRef
31.
Zurück zum Zitat Wang X, Dong L, Yan J (2012) Maximum ambiguity based sample selection in fuzzy decision tree induction. IEEE Trans Knowl Data Eng 24(8):1491–1505CrossRef Wang X, Dong L, Yan J (2012) Maximum ambiguity based sample selection in fuzzy decision tree induction. IEEE Trans Knowl Data Eng 24(8):1491–1505CrossRef
32.
Zurück zum Zitat Wang X, Dong C (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567CrossRef Wang X, Dong C (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567CrossRef
33.
Zurück zum Zitat Sharma A, Imoto S, Miyano S (2012) A filter based feature selection algorithm using null space of covariance matrix for DNA microarray gene expression data. Curr Bioinform 7(3):289–294CrossRef Sharma A, Imoto S, Miyano S (2012) A filter based feature selection algorithm using null space of covariance matrix for DNA microarray gene expression data. Curr Bioinform 7(3):289–294CrossRef
34.
Metadaten
Titel
Principal component analysis using QR decomposition
verfasst von
Alok Sharma
Kuldip K. Paliwal
Seiya Imoto
Satoru Miyano
Publikationsdatum
01.12.2013
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 6/2013
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-012-0131-7

Weitere Artikel der Ausgabe 6/2013

International Journal of Machine Learning and Cybernetics 6/2013 Zur Ausgabe

Neuer Inhalt