Abstract
Principal components analysis is an important and well-studied subject in statistics and signal processing. Several algorithms for solving this problem exist, and could be mostly grouped into one of the following three approaches: adaptation based on Hebbian updates and deflation, optimization of a second order statistical criterion (like reconstruction error or output variance), and fixed point update rules with deflation. In this study, we propose an alternate approach that avoids deflation and gradient-search techniques. The proposed method is an on-line procedure based on recursively updating the eigenvector and eigenvalue matrices with every new sample such that the estimates approximately track their true values as would be calculated analytically from the current sample estimate of the data covariance matrix. The perturbation technique is theoretically shown to be applicable for recursive canonical correlation analysis, as well. The performance of this algorithm is compared with that of a structurally similar matrix perturbation-based method and also with a few other traditional methods like Sanger’s rule and APEX.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
R.O. Duda and P.E. Hart, “Pattern Classification and Scene Analysis,” Wiley, New York, 1973.
S.Y. Kung, K.I. Diamantaras, and J.S. Taur, “Adaptive Principal Component Extraction (APEX) and Applications,” IEEE Trans. Signal Process., vol. 42, no. 5, 1994, pp. 1202–1217.
J. Mao and A.K. Jain, “Artificial Neural Networks for Feature Extraction and Multivariate Data Projection,” IEEE Trans. Neural Netw., vol. 6, no. 2, 1995, pp. 296–317.
Y. Cao, S. Sridharan, and M. Moody, “Multichannel Speech Separation by Eigendecomposition and its Application to Co-Talker Interference Removal,” IEEE Trans. Speech Audio Process., vol. 5, no. 3, 1997, pp. 209–219.
G. Golub and C.V. Loan, “Matrix Computation,” Johns Hopkins University Press, Baltimore, Maryland, 1993.
E. Oja, “Subspace Methods for Pattern Recognition,” Wiley, New York, 1983.
T.D. Sanger, “Optimal Unsupervised Learning in a Single Layer Linear Feedforward Neural Network,” Neural Netw., vol. 2, no. 6, 1989, pp. 459–473.
J. Rubner and K. Schulten, “Development of Feature Detectors by Self Organization,” Biol. Cybern., vol. 62, 1990, pp. 193–199.
J. Rubner and P. Tavan, “A Self Organizing Network for Principal Component Analysis,” Europhys. Lett., vol. 10, 1989, pp. 693–698.
L. Xu, “Least Mean Square Error Reconstruction Principle for Self-Organizing Neural-Nets,” Neural Netw., vol. 6, 1993, pp. 627–648.
B. Yang, “Projection Approximation Subspace Tracking,” IEEE Trans. Signal Process., vol. 43, no. 1, 1995, pp. 95–107.
Y. Hua, Y. Xiang, T. Chen, K. Abed-Meriam, and Y. Miao, “Natural Power Method for Fast Subspace Tracking,” in Proceedings of NNSP’99, 1999, pp. 176–185.
Y.N. Rao and J.C. Principe, “Robust On-line Principal Component Analysis Based on a Fixed-Point Approach,” in Proceedings of ICASSP’02, vol. 1, 2002, pp. 981–984.
D. Erdogmus, Y.N. Rao, K.E. Hild II, and J.C. Principe, “Simultaneous Principal Component Extraction with Application to Adaptive Blind Multiuser Detection,” EURASTP J. Appl. Signal Process., vol. 2002, no. 12, 2002, pp. 1473–1484.
B. Champagne, “Adaptive Eigendecomposition of Data Covariance Matrices Based on First-Order Perturbations,” IEEE Trans. Signal Process., vol. 42, no. 10, 1994, pp. 2758–2770.
J. Via, I. Santamaria, and J. Perez, “A Robust RLS Algorithm for Adaptive Canonical Correlation Analysis,” in Proceedings of ICASSP’05, Philadelphia, Pennsylvania, 2005.
S. Haykin, “Adaptive Filter Theory,” 4th ed., Prentice Hall, Upper Saddle River, New Jersey, 2001, pp. 231–319.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Hegde, A., Principe, J.C., Erdogmus, D. et al. Perturbation-Based Eigenvector Updates for On-Line Principal Components Analysis and Canonical Correlation Analysis. J VLSI Sign Process Syst Sign Image Video Technol 45, 85–95 (2006). https://doi.org/10.1007/s11265-006-9773-6
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11265-006-9773-6