Skip to main content

2017 | OriginalPaper | Buchkapitel

9. Singular Feature Extraction and Its Neural Networks

verfasst von : Xiangyu Kong, Changhua Hu, Zhansheng Duan

Erschienen in: Principal Component Analysis Networks and Algorithms

Verlag: Springer Singapore

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

From the preceding chapters, we have seen that in the wake of the important initiative work by Oja and Sanger, many neural network learning algorithms for PCA have been developed.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kaiser, A., Schenck, M., & Moller, R. (2010). Coupled singular value decomposition of a cross-covariance matrix. International Journal of Neural Systems, 20(4), 293–318.CrossRef Kaiser, A., Schenck, M., & Moller, R. (2010). Coupled singular value decomposition of a cross-covariance matrix. International Journal of Neural Systems, 20(4), 293–318.CrossRef
2.
Zurück zum Zitat Feng, D. Z., Zhang, X. D., & Bao, Z. (2004). A neural network learning for adaptively extracting cross-correlation features between two high-dimensional data streams. IEEE Transactions on Neural Networks, 15(6), 1541–1554.CrossRef Feng, D. Z., Zhang, X. D., & Bao, Z. (2004). A neural network learning for adaptively extracting cross-correlation features between two high-dimensional data streams. IEEE Transactions on Neural Networks, 15(6), 1541–1554.CrossRef
3.
4.
Zurück zum Zitat Comon, P., & Golub, G. H. (1990). Tracking a few extreme singular values and vectors in signal processing. Processing of the IEEE, 78(8), 1318–1343. Comon, P., & Golub, G. H. (1990). Tracking a few extreme singular values and vectors in signal processing. Processing of the IEEE, 78(8), 1318–1343.
5.
Zurück zum Zitat Ferali, W., & Proakis, J. G. (1990). Adaptive SVD algorithm for covariance matrix eigenstructure computation. In Proceedings of 1990 IEEE International Conference on Acoustic, Speech and Signal Processing (pp. 176–179). Ferali, W., & Proakis, J. G. (1990). Adaptive SVD algorithm for covariance matrix eigenstructure computation. In Proceedings of 1990 IEEE International Conference on Acoustic, Speech and Signal Processing (pp. 176–179).
6.
Zurück zum Zitat Mooden, M., Dooren, P. V., & Vandewalle, J. (1992). A singular value decomposition updating algorithm for subspace tracking. SIAM Journal of Matrix Analysis and Applications, 13(4), 1015–1038.MathSciNetCrossRefMATH Mooden, M., Dooren, P. V., & Vandewalle, J. (1992). A singular value decomposition updating algorithm for subspace tracking. SIAM Journal of Matrix Analysis and Applications, 13(4), 1015–1038.MathSciNetCrossRefMATH
7.
Zurück zum Zitat Yuile, A. L., Kammen, D. M., & Cohen, D. S. (1985). Quadrature and the development of orientation selective cortical cells by Hebb rules. Biology Cybernetics, 61(3), 183–194.CrossRefMATH Yuile, A. L., Kammen, D. M., & Cohen, D. S. (1985). Quadrature and the development of orientation selective cortical cells by Hebb rules. Biology Cybernetics, 61(3), 183–194.CrossRefMATH
8.
Zurück zum Zitat Samardzija, N., & Waterland, R. L. (1989). A neural network for computing eigenvectors and eigenvalues. Biology Cybernetics, 65(4), 211–214.MathSciNetCrossRefMATH Samardzija, N., & Waterland, R. L. (1989). A neural network for computing eigenvectors and eigenvalues. Biology Cybernetics, 65(4), 211–214.MathSciNetCrossRefMATH
9.
Zurück zum Zitat Cichocki, A., & Unbehauen, R. (1992). Neural network for computing eigenvalues and eigenvectors. Biology Cybernetics, 68(2), 155–159.CrossRefMATH Cichocki, A., & Unbehauen, R. (1992). Neural network for computing eigenvalues and eigenvectors. Biology Cybernetics, 68(2), 155–159.CrossRefMATH
10.
Zurück zum Zitat Cichocki, A. (1992). Neural network for singular value decomposition. Electronic Letter, 28(8), 784–786.CrossRef Cichocki, A. (1992). Neural network for singular value decomposition. Electronic Letter, 28(8), 784–786.CrossRef
11.
Zurück zum Zitat Cichocki, A., & Unbehauen, R. (1994). Neural networks for optimization and signal processing. New York: Wiley.MATH Cichocki, A., & Unbehauen, R. (1994). Neural networks for optimization and signal processing. New York: Wiley.MATH
12.
Zurück zum Zitat Diamantaras, K. I., & Kung, S. Y. (1994). Cross-correlation neural network models. IEEE Transactions on Signal Processing, 42(11), 3218–3223.CrossRef Diamantaras, K. I., & Kung, S. Y. (1994). Cross-correlation neural network models. IEEE Transactions on Signal Processing, 42(11), 3218–3223.CrossRef
13.
Zurück zum Zitat Feng, D. Z., Bao, Z., & Zhang, X. D. (2001). A cross-associative neural network for SVD of nonsquared data matrix in signal processing. IEEE Transactions on Neural Networks, 12(9), 1215–1221.CrossRef Feng, D. Z., Bao, Z., & Zhang, X. D. (2001). A cross-associative neural network for SVD of nonsquared data matrix in signal processing. IEEE Transactions on Neural Networks, 12(9), 1215–1221.CrossRef
14.
Zurück zum Zitat Feng, D. Z., & Bao, Z. (1997). Modified cross-correlation neural networks. Chinese Journal of Electronic, 6(3), 63–70. Feng, D. Z., & Bao, Z. (1997). Modified cross-correlation neural networks. Chinese Journal of Electronic, 6(3), 63–70.
15.
Zurück zum Zitat Sanger, T. D. (1994).Two iterative algorithms for computing the singular value decomposition from input/output samples. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, Vol. 6 (pp. 144–151). Morgan Kaufmann. Sanger, T. D. (1994).Two iterative algorithms for computing the singular value decomposition from input/output samples. In J. D. Cowan, G. Tesauro, & J. Alspector (Eds.), Advances in neural information processing systems, Vol. 6 (pp. 144–151). Morgan Kaufmann.
16.
Zurück zum Zitat Diamantaras, K. I., & Kung, S. Y. (1994). Multilayer neural networks for reduced-rank approximation. IEEE Transactions on Neural Networks, 5(5), 684–697.CrossRef Diamantaras, K. I., & Kung, S. Y. (1994). Multilayer neural networks for reduced-rank approximation. IEEE Transactions on Neural Networks, 5(5), 684–697.CrossRef
17.
Zurück zum Zitat Fiori, S. (2003). Singular value decomposition learning on double Stiefel manifolds. International Journal of Neural Systems, 13(2), 1–16. Fiori, S. (2003). Singular value decomposition learning on double Stiefel manifolds. International Journal of Neural Systems, 13(2), 1–16.
18.
Zurück zum Zitat Weingessel, A., & Hornik, K. (1997). SVD algorithms: APEX-like versus subspace methods. Neural Processing Letter, 5(3), 177–184.CrossRef Weingessel, A., & Hornik, K. (1997). SVD algorithms: APEX-like versus subspace methods. Neural Processing Letter, 5(3), 177–184.CrossRef
19.
Zurück zum Zitat Weingessel, A. (1999). An analysis of learning algorithms in PCA and SVD Neural Networks. PhD thesis, Technische University at Wien. Weingessel, A. (1999). An analysis of learning algorithms in PCA and SVD Neural Networks. PhD thesis, Technische University at Wien.
20.
Zurück zum Zitat Helmke, U., & Moore, J. B. (1992). Singular-value decomposition via gradient and self-equivalent flows. Linear Algebra and Application, 169(92), 223–248.MathSciNetCrossRefMATH Helmke, U., & Moore, J. B. (1992). Singular-value decomposition via gradient and self-equivalent flows. Linear Algebra and Application, 169(92), 223–248.MathSciNetCrossRefMATH
21.
Zurück zum Zitat Kong, X. Y., Ma, H. G., An, Q. S., & Zhang, Q. (2015). An effective neural learning algorithm for extracting cross-correlation feature between two high-dimensional data streams. Neural Processing Letter, 42(2), 459–477.CrossRef Kong, X. Y., Ma, H. G., An, Q. S., & Zhang, Q. (2015). An effective neural learning algorithm for extracting cross-correlation feature between two high-dimensional data streams. Neural Processing Letter, 42(2), 459–477.CrossRef
22.
Zurück zum Zitat Feng, X. W., Kong, X. Y., & Ma, H. G. (2016). Coupled cross-correlation neural network algorithm for principal singular triplet extraction of a cross-covariance matrix. IEEE/CAA Journal of Automatica Sinica, 3(2), 141–148.MathSciNetCrossRef Feng, X. W., Kong, X. Y., & Ma, H. G. (2016). Coupled cross-correlation neural network algorithm for principal singular triplet extraction of a cross-covariance matrix. IEEE/CAA Journal of Automatica Sinica, 3(2), 141–148.MathSciNetCrossRef
23.
Zurück zum Zitat Brockett, R. W. (1991). Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Linear Algebra and its Applications, 146, 79–91.MathSciNetCrossRefMATH Brockett, R. W. (1991). Dynamical systems that sort lists, diagonalize matrices and solve linear programming problems. Linear Algebra and its Applications, 146, 79–91.MathSciNetCrossRefMATH
24.
Zurück zum Zitat Yang, B. (1995). Projection approximation subspace tracking. IEEE Transactions on Signal Processing, 43(1), 95–107.CrossRef Yang, B. (1995). Projection approximation subspace tracking. IEEE Transactions on Signal Processing, 43(1), 95–107.CrossRef
25.
Zurück zum Zitat Golub, G. H., & Van Loan, C. F. (1989). Matrix computations. Baltimore, MD: The Johns Hopkins University Press.MATH Golub, G. H., & Van Loan, C. F. (1989). Matrix computations. Baltimore, MD: The Johns Hopkins University Press.MATH
26.
Zurück zum Zitat Haykin, S. (1991). Adaptive filter theory (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.MATH Haykin, S. (1991). Adaptive filter theory (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.MATH
27.
Zurück zum Zitat Wax, M., & Kailath, T. (1985). Detection of signals by information theoretic criteria. IEEE Transactions on Acoustic, Speech and Signal Processing, 33(2), 387–392.MathSciNetCrossRef Wax, M., & Kailath, T. (1985). Detection of signals by information theoretic criteria. IEEE Transactions on Acoustic, Speech and Signal Processing, 33(2), 387–392.MathSciNetCrossRef
28.
Zurück zum Zitat Schmidt R. (1986). Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas Propagation, AP-34(3), 276–281. Schmidt R. (1986). Multiple emitter location and signal parameter estimation. IEEE Transactions on Antennas Propagation, AP-34(3), 276–281.
29.
Zurück zum Zitat Roy, R., Paulraj, A., & Kailath, T. (1986). ESPRIT-A subspace rotation approach to estimation of parameters of cissoids in noise. IEEE Transactions on Acoustic, Speech and Signal Processing, 34(10), 1340–1342.CrossRef Roy, R., Paulraj, A., & Kailath, T. (1986). ESPRIT-A subspace rotation approach to estimation of parameters of cissoids in noise. IEEE Transactions on Acoustic, Speech and Signal Processing, 34(10), 1340–1342.CrossRef
30.
Zurück zum Zitat Kong, X. Y., Hu, C. H., & Han, C. Z. (2012). A dual purpose principal and minor subspace gradient flow. IEEE Transactions on Signal Processing, 60(1), 197–210.MathSciNetCrossRef Kong, X. Y., Hu, C. H., & Han, C. Z. (2012). A dual purpose principal and minor subspace gradient flow. IEEE Transactions on Signal Processing, 60(1), 197–210.MathSciNetCrossRef
31.
Zurück zum Zitat Magnus, J. R., & Neudecker, H. (1991). Matrix differential calculus with applications in statistics and econometrics (2nd ed.). New York: Wiley.MATH Magnus, J. R., & Neudecker, H. (1991). Matrix differential calculus with applications in statistics and econometrics (2nd ed.). New York: Wiley.MATH
32.
Zurück zum Zitat Kushner, H. J., & Clark, D. S. (1976). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer-Verlag. Kushner, H. J., & Clark, D. S. (1976). Stochastic approximation methods for constrained and unconstrained systems. New York: Springer-Verlag.
33.
34.
Zurück zum Zitat Moller, R., & Konies, A. (2004). Coupled principal component analysis. IEEE Transactions on Neural Networks, 15(1), 214–222.CrossRef Moller, R., & Konies, A. (2004). Coupled principal component analysis. IEEE Transactions on Neural Networks, 15(1), 214–222.CrossRef
35.
Zurück zum Zitat Noble, B., & Daniel, J. W. (1988). Applied linear algebra (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.MATH Noble, B., & Daniel, J. W. (1988). Applied linear algebra (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.MATH
36.
37.
Zurück zum Zitat Hotelling, H. (1943). Further points on matrix calculation and simultaneous equations. The Annals of Mathematical Statistics, 14(4), 440–441.MathSciNetCrossRefMATH Hotelling, H. (1943). Further points on matrix calculation and simultaneous equations. The Annals of Mathematical Statistics, 14(4), 440–441.MathSciNetCrossRefMATH
38.
Zurück zum Zitat Moller, R., & Homann, H. (2004). An extension of neural gas to local PCA. Neurocomputing, 62(1), 305–326. Moller, R., & Homann, H. (2004). An extension of neural gas to local PCA. Neurocomputing, 62(1), 305–326.
39.
Zurück zum Zitat Golub, G. H., & Reinsch, C. (1970). Singular value decomposition and least squares solutions. Numerical Mathematics, 14(5), 403–420.CrossRefMATH Golub, G. H., & Reinsch, C. (1970). Singular value decomposition and least squares solutions. Numerical Mathematics, 14(5), 403–420.CrossRefMATH
40.
Zurück zum Zitat Moller, R. (2006). First-order approximation of Gram-Schmidt orthonormalization beats deflation in coupled PCA learning rules. Neurocomputing, 69(13–15), 1582–1590.CrossRef Moller, R. (2006). First-order approximation of Gram-Schmidt orthonormalization beats deflation in coupled PCA learning rules. Neurocomputing, 69(13–15), 1582–1590.CrossRef
41.
Zurück zum Zitat Baldi, P. F., & Hornik, K. (1995). Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks, 6(4), 837–858.CrossRef Baldi, P. F., & Hornik, K. (1995). Learning in linear neural networks: A survey. IEEE Transactions on Neural Networks, 6(4), 837–858.CrossRef
42.
Zurück zum Zitat Hasan, M. (2008). A logarithmic cost function for principal singular component analysis. In IEEE International Conference on Acoustic Speech and Signal Processing (pp. 1933–1936). Hasan, M. (2008). A logarithmic cost function for principal singular component analysis. In IEEE International Conference on Acoustic Speech and Signal Processing (pp. 1933–1936).
43.
Zurück zum Zitat Haykin, S. (1999). Neural networks. A comprehensive foundation (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.MATH Haykin, S. (1999). Neural networks. A comprehensive foundation (2nd ed.). Englewood Cliffs, NJ: Prentice Hall.MATH
44.
Zurück zum Zitat Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(1), 533–536.CrossRef Rumelhart, D. E., Hinton, G. E., & Williams, R. J. (1986). Learning representations by back-propagating errors. Nature, 323(1), 533–536.CrossRef
45.
Zurück zum Zitat Baldi, P. F., & Hornik, K. (1989). Neural networks and principal component analysis: learning from examples without local minima. Neural Networks, 2(1), 53–58.CrossRef Baldi, P. F., & Hornik, K. (1989). Neural networks and principal component analysis: learning from examples without local minima. Neural Networks, 2(1), 53–58.CrossRef
46.
Zurück zum Zitat Smith, S. T. (1991). Dynamical systems that perform the singular value decomposition. System Control Letter, 16(5), 319–327.MathSciNetCrossRefMATH Smith, S. T. (1991). Dynamical systems that perform the singular value decomposition. System Control Letter, 16(5), 319–327.MathSciNetCrossRefMATH
47.
Zurück zum Zitat Hori, G. (2003). A general framework for SVD flows and joint SVD flows. In Proceeding of IEEE International Conference on Acoustic Speech and Signal Processing, Vol. 2. (pp. 693–696). Hori, G. (2003). A general framework for SVD flows and joint SVD flows. In Proceeding of IEEE International Conference on Acoustic Speech and Signal Processing, Vol. 2. (pp. 693–696).
48.
Zurück zum Zitat Hasan, M. A. (2008). Low-rank approximations with applications to principal singular component learning systems. In Proceedings of 2008 IEEE Conference on Decision and Control (pp. 3293–3298). Hasan, M. A. (2008). Low-rank approximations with applications to principal singular component learning systems. In Proceedings of 2008 IEEE Conference on Decision and Control (pp. 3293–3298).
49.
Zurück zum Zitat Luo, F. L., Unbehauen, R., & Li, Y. D. (1997). Real-time computation of singular vectors. Application Mathematics and Computation, 86(2–3), 197–214.MathSciNetCrossRefMATH Luo, F. L., Unbehauen, R., & Li, Y. D. (1997). Real-time computation of singular vectors. Application Mathematics and Computation, 86(2–3), 197–214.MathSciNetCrossRefMATH
50.
Zurück zum Zitat Chong, E. K. P., & Zak, S. H. (2001). An introduction to optimization (2nd ed.). New York: Wiley.MATH Chong, E. K. P., & Zak, S. H. (2001). An introduction to optimization (2nd ed.). New York: Wiley.MATH
51.
Zurück zum Zitat Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993). Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4), 558–569.CrossRef Martinetz, T. M., Berkovich, S. G., & Schulten, K. J. (1993). Neural-gas’ network for vector quantization and its application to time-series prediction. IEEE Transactions on Neural Networks, 4(4), 558–569.CrossRef
52.
Zurück zum Zitat Brockett, R. W. (1991). Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications, 146, 79–91.MathSciNetCrossRefMATH Brockett, R. W. (1991). Dynamical systems that sort lists, diagonalize matrices, and solve linear programming problems. Linear Algebra and its Applications, 146, 79–91.MathSciNetCrossRefMATH
Metadaten
Titel
Singular Feature Extraction and Its Neural Networks
verfasst von
Xiangyu Kong
Changhua Hu
Zhansheng Duan
Copyright-Jahr
2017
Verlag
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-2915-8_9