Skip to main content
Erschienen in: Soft Computing 4/2012

01.04.2012 | Original Paper

A modified learning algorithm for the multilayer neural network with multi-valued neurons based on the complex QR decomposition

verfasst von: Igor Aizenberg, Antonio Luchetta, Stefano Manetti

Erschienen in: Soft Computing | Ausgabe 4/2012

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, a modified learning algorithm for the multilayer neural network with the multi-valued neurons (MLMVN) is presented. The MLMVN, which is a member of complex-valued neural networks family, has already demonstrated a number of important advantages over other techniques. A modified learning algorithm for this network is based on the introduction of an acceleration step, performing by means of the complex QR decomposition and on the new approach to calculation of the output neurons errors: they are calculated as the differences between the corresponding desired outputs and actual values of the weighted sums. These modifications significantly improve the existing derivative-free backpropagation learning algorithm for the MLMVN in terms of learning speed. A modified learning algorithm requires two orders of magnitude lower number of training epochs and less time for its convergence when compared with the existing learning algorithm. Good performance is confirmed not only by the much quicker convergence of the learning algorithm, but also by the compatible or even higher classification/prediction accuracy, which is obtained by testing over some benchmarks (Mackey–Glass and Jenkins–Box time series) and over some satellite spectral data examined in a comparison test.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Aizenberg I (2010) A periodic activation function and a modified learning algorithm for a multi-valued neuron. IEEE Trans Neural Netw 21(12):1939–1949CrossRef Aizenberg I (2010) A periodic activation function and a modified learning algorithm for a multi-valued neuron. IEEE Trans Neural Netw 21(12):1939–1949CrossRef
Zurück zum Zitat Aizenberg I (2011) Complex-valued neural networks with multi-valued neurons. Springer, HeidelbergMATHCrossRef Aizenberg I (2011) Complex-valued neural networks with multi-valued neurons. Springer, HeidelbergMATHCrossRef
Zurück zum Zitat Aizenberg NN, Aizenberg IN (1992) CNN based on multi-valued neuron as a model of associative memory for gray-scale images. Proceedings of the Second IEEE International Workshop on Cellular Neural Networks and their Applications. Technical University Munich, Germany, October 14–16, pp 36–41 Aizenberg NN, Aizenberg IN (1992) CNN based on multi-valued neuron as a model of associative memory for gray-scale images. Proceedings of the Second IEEE International Workshop on Cellular Neural Networks and their Applications. Technical University Munich, Germany, October 14–16, pp 36–41
Zurück zum Zitat Aizenberg NN, Ivaskiv YL (1977) Multiple-valued threshold logic. Naukova Dumka Publisher House, Kiev (in Russian) Aizenberg NN, Ivaskiv YL (1977) Multiple-valued threshold logic. Naukova Dumka Publisher House, Kiev (in Russian)
Zurück zum Zitat Aizenberg I, Moraga C (2007) Multilayer feedforward neural network based on multi-valued neurons (MLMVN) and a backpropagation learning algorithm. Soft Comput 11(2):169–183 Aizenberg I, Moraga C (2007) Multilayer feedforward neural network based on multi-valued neurons (MLMVN) and a backpropagation learning algorithm. Soft Comput 11(2):169–183
Zurück zum Zitat Aizenberg I, Moraga C (2007) The genetic code as a function of multiple-valued logic over the field of complex numbers and its learning using multilayer neural network based on multi-valued neurons. J Multiple-Valued Logic Soft Comput (4–6):605–618 Aizenberg I, Moraga C (2007) The genetic code as a function of multiple-valued logic over the field of complex numbers and its learning using multilayer neural network based on multi-valued neurons. J Multiple-Valued Logic Soft Comput (4–6):605–618
Zurück zum Zitat Aizenberg I, Zurada J (2007) Solving selected classification problems in bioinformatics using multilayer neural network based on multi-valued neurons (MLMVN). In: Marques de Sá J et al (eds) Proceedings of the International Conference on Artificial Neural Networks (ICANN-2007), Lecture Notes in Computer Science, vol 4668, Part I. Springer, Berlin, Heidelberg, New York, pp 874–883 Aizenberg I, Zurada J (2007) Solving selected classification problems in bioinformatics using multilayer neural network based on multi-valued neurons (MLMVN). In: Marques de Sá J et al (eds) Proceedings of the International Conference on Artificial Neural Networks (ICANN-2007), Lecture Notes in Computer Science, vol 4668, Part I. Springer, Berlin, Heidelberg, New York, pp 874–883
Zurück zum Zitat Aizenberg NN, Ivaskiv YL, Pospelov DA (1971) About one generalization of the threshold function. Doklady Akademii Nauk SSSR (The Reports of the Academy of Sciences of the USSR) (in Russian) 196(6):1287–1290 Aizenberg NN, Ivaskiv YL, Pospelov DA (1971) About one generalization of the threshold function. Doklady Akademii Nauk SSSR (The Reports of the Academy of Sciences of the USSR) (in Russian) 196(6):1287–1290
Zurück zum Zitat Aizenberg I, Aizenberg N, Vandewalle J (2000) Multi-valued and universal binary neurons theory, learning and applications. Kluwer Academic Publishers, Boston/Dordecht/London Aizenberg I, Aizenberg N, Vandewalle J (2000) Multi-valued and universal binary neurons theory, learning and applications. Kluwer Academic Publishers, Boston/Dordecht/London
Zurück zum Zitat Aizenberg I, Moraga C, Paliy D (2005) Feedforward neural network based on multi-valued neurons. In: Reusch B (ed) Computational intelligence, theory and applications. Advances in Soft Computing, XIV. Springer, Berlin, pp 599–612 Aizenberg I, Moraga C, Paliy D (2005) Feedforward neural network based on multi-valued neurons. In: Reusch B (ed) Computational intelligence, theory and applications. Advances in Soft Computing, XIV. Springer, Berlin, pp 599–612
Zurück zum Zitat Aizenberg I, Paliy D, Zurada J, Astola J (2008) Blur identification by multilayer neural network based on multivalued neurons. IEEE Trans Neural Netw 19(5):883–898 Aizenberg I, Paliy D, Zurada J, Astola J (2008) Blur identification by multilayer neural network based on multivalued neurons. IEEE Trans Neural Netw 19(5):883–898
Zurück zum Zitat Amato U, Masiello G, Serio C, Viggiano M (2002) The σ-IASI code for calculation of infrared atmosphere radiance and its derivatives. Environ Model Softw 17:651–667CrossRef Amato U, Masiello G, Serio C, Viggiano M (2002) The σ-IASI code for calculation of infrared atmosphere radiance and its derivatives. Environ Model Softw 17:651–667CrossRef
Zurück zum Zitat Aoki H, Kosugi Y (2000) An image storage system using complex-valued associative memory. Proceedings of the 15th International Conference on Pattern Recognition, vol 2, IEEE Computer Society Press, Barcelona, pp 626–629 Aoki H, Kosugi Y (2000) An image storage system using complex-valued associative memory. Proceedings of the 15th International Conference on Pattern Recognition, vol 2, IEEE Computer Society Press, Barcelona, pp 626–629
Zurück zum Zitat Aoki H, Watanabe E, Nagata A, Kosugi Y (2001) Rotation-invariant image association for endoscopic positional identification using complex-valued associative memories. In: Mira J, Prieto A (eds) Bio-inspired applications of connectionism, Lecture notes in computer science, vol 2085. Springer, Berlin, pp 369–374 Aoki H, Watanabe E, Nagata A, Kosugi Y (2001) Rotation-invariant image association for endoscopic positional identification using complex-valued associative memories. In: Mira J, Prieto A (eds) Bio-inspired applications of connectionism, Lecture notes in computer science, vol 2085. Springer, Berlin, pp 369–374
Zurück zum Zitat Box GEP, Jenkins GM (1976) Time series analysis, forecasting and control. Holden Day, San FranciscoMATH Box GEP, Jenkins GM (1976) Time series analysis, forecasting and control. Holden Day, San FranciscoMATH
Zurück zum Zitat Cannas B, Fanni A, Manetti S, Montisci A, Piccirilli M (2004) Neural network based analog fault diagnosis using testability analysis. Neural Comput Appl 13(4):288–298CrossRef Cannas B, Fanni A, Manetti S, Montisci A, Piccirilli M (2004) Neural network based analog fault diagnosis using testability analysis. Neural Comput Appl 13(4):288–298CrossRef
Zurück zum Zitat Cervellera C, Maccio D, Muselli M (2008) Deterministic learning for maximum-likelihood estimation through neural networks. IEEE Trans Neural Netw 19(8):1456–1467CrossRef Cervellera C, Maccio D, Muselli M (2008) Deterministic learning for maximum-likelihood estimation through neural networks. IEEE Trans Neural Netw 19(8):1456–1467CrossRef
Zurück zum Zitat Chen F, Chen G, He G, Xu X, He Q (2009) Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations. IEEE Trans Neural Netw 20(10):1645–1658CrossRef Chen F, Chen G, He G, Xu X, He Q (2009) Universal perceptron and DNA-like learning algorithm for binary neural networks: LSBF and PBF implementations. IEEE Trans Neural Netw 20(10):1645–1658CrossRef
Zurück zum Zitat Fedi G, Manetti S, Pelosi G, Selleri S (2001) Profiled corrugated circular horns analysis and synthesis via an artificial neural network. IEEE Trans Antennas Propag 49(11):1597–1602CrossRef Fedi G, Manetti S, Pelosi G, Selleri S (2001) Profiled corrugated circular horns analysis and synthesis via an artificial neural network. IEEE Trans Antennas Propag 49(11):1597–1602CrossRef
Zurück zum Zitat Fiori S (2008) Learning by criterion optimization on a unitary unimodular matrix group. J Neural Syst 18(2):87–103MathSciNetCrossRef Fiori S (2008) Learning by criterion optimization on a unitary unimodular matrix group. J Neural Syst 18(2):87–103MathSciNetCrossRef
Zurück zum Zitat Furao S, Ogura T, Hasegawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20(8):893–903MATHCrossRef Furao S, Ogura T, Hasegawa O (2007) An enhanced self-organizing incremental neural network for online unsupervised learning. Neural Netw 20(8):893–903MATHCrossRef
Zurück zum Zitat Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, New York Golub GH, Van Loan CF (1996) Matrix computations, 3rd edn. Johns Hopkins University Press, New York
Zurück zum Zitat Grieco G, Luchetta A, Masiello G, Serio C, Viaggiano M (2005) IMG O3 retrieval and comparison with TOMS/ADEOS columnar ozone: an analysis based on tropical soundings”. J Quant Spectrosc Radiat Transf 95(3):331–348CrossRef Grieco G, Luchetta A, Masiello G, Serio C, Viaggiano M (2005) IMG O3 retrieval and comparison with TOMS/ADEOS columnar ozone: an analysis based on tropical soundings”. J Quant Spectrosc Radiat Transf 95(3):331–348CrossRef
Zurück zum Zitat Gupta MM, Jin L, Homma N (2003) Static and dynamic neural networks. Wiley, Hoboken/New JerseyCrossRef Gupta MM, Jin L, Homma N (2003) Static and dynamic neural networks. Wiley, Hoboken/New JerseyCrossRef
Zurück zum Zitat Haykin S (1999) Neural networks: a comprehensive foundation. Prentice Hall, Englewood Cliffs Haykin S (1999) Neural networks: a comprehensive foundation. Prentice Hall, Englewood Cliffs
Zurück zum Zitat Jankowski S, Lozowski A, Zurada JM (1996) Complex-valued multistate neural associative memory. IEEE Trans Neural Netw 7:1491–1496CrossRef Jankowski S, Lozowski A, Zurada JM (1996) Complex-valued multistate neural associative memory. IEEE Trans Neural Netw 7:1491–1496CrossRef
Zurück zum Zitat Jin N, Liu D (2008) Wavelet basis function neural networks for sequential learning. IEEE Trans Neural Netw 19(3):523–528CrossRef Jin N, Liu D (2008) Wavelet basis function neural networks for sequential learning. IEEE Trans Neural Netw 19(3):523–528CrossRef
Zurück zum Zitat Lawson CL, Hanson RJ (1974) Solving least squares problems. Prentice-Hall, Englewood Cliffs Lawson CL, Hanson RJ (1974) Solving least squares problems. Prentice-Hall, Englewood Cliffs
Zurück zum Zitat Luchetta A (2008) Automatic generation of the optimum threshold for parameter weighted pruning in multiple heterogeneous output neural networks. Neurocomputing 71(16–18):3553–3560CrossRef Luchetta A (2008) Automatic generation of the optimum threshold for parameter weighted pruning in multiple heterogeneous output neural networks. Neurocomputing 71(16–18):3553–3560CrossRef
Zurück zum Zitat Luchetta A, Manetti S, Francini F (1998) Forecast: a neural system for diagnosis and control of highway surfaces. IEEE Intell Syst 13(3):20–26CrossRef Luchetta A, Manetti S, Francini F (1998) Forecast: a neural system for diagnosis and control of highway surfaces. IEEE Intell Syst 13(3):20–26CrossRef
Zurück zum Zitat Luchetta A, Serio C, Viggiano M (2005) A soft computing approach to the elaboration of satellite data. Proceedings of the 2005 IEEE International Workshop on Soft Computing Applications (SOFA2005), Szeged, Hungary and Arad, Romania Luchetta A, Serio C, Viggiano M (2005) A soft computing approach to the elaboration of satellite data. Proceedings of the 2005 IEEE International Workshop on Soft Computing Applications (SOFA2005), Szeged, Hungary and Arad, Romania
Zurück zum Zitat Luchetta A, Manetti S, Pellegrini L, Pelosi G, Selleri S (2006) Design of waveguide microwave filters by means of artificial neural networks. Int J RF Microw Computer-Aided Eng 16(6):554–560CrossRef Luchetta A, Manetti S, Pellegrini L, Pelosi G, Selleri S (2006) Design of waveguide microwave filters by means of artificial neural networks. Int J RF Microw Computer-Aided Eng 16(6):554–560CrossRef
Zurück zum Zitat Mackey M, Glass L (1977) Oscillation and chaos in physiological control systems. Science 197:287–289CrossRef Mackey M, Glass L (1977) Oscillation and chaos in physiological control systems. Science 197:287–289CrossRef
Zurück zum Zitat Mandic D, Su Lee Goh V (2009) Complex valued nonlinear adaptive filters noncircularity. Widely linear and neural models. Wiley, New YorkCrossRef Mandic D, Su Lee Goh V (2009) Complex valued nonlinear adaptive filters noncircularity. Widely linear and neural models. Wiley, New YorkCrossRef
Zurück zum Zitat Manetti S, Luchetta A (2003) A real time hydrological forecasting system using a fuzzy clustering approach. Comput Geosci 29(9):1111–1117CrossRef Manetti S, Luchetta A (2003) A real time hydrological forecasting system using a fuzzy clustering approach. Comput Geosci 29(9):1111–1117CrossRef
Zurück zum Zitat Muezzinoglu MK, Guzelis C, Zurada JM (2003) A new design method for the complex-valued multistate Hopfield associative memory. IEEE Trans Neural Netw 14(4):891–899CrossRef Muezzinoglu MK, Guzelis C, Zurada JM (2003) A new design method for the complex-valued multistate Hopfield associative memory. IEEE Trans Neural Netw 14(4):891–899CrossRef
Zurück zum Zitat Peng J, Li K, Irwin GW (2008) A new Jacobian matrix for optimal learning of single-layer neural networks. IEEE Trans Neural Netw 19(1):119–129CrossRef Peng J, Li K, Irwin GW (2008) A new Jacobian matrix for optimal learning of single-layer neural networks. IEEE Trans Neural Netw 19(1):119–129CrossRef
Zurück zum Zitat Rumelhart DE, McClelland JL (1986) Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge Rumelhart DE, McClelland JL (1986) Parallel distributed processing: explorations in the microstructure of cognition. MIT Press, Cambridge
Zurück zum Zitat Stoer J, Bulirsch R (1991) Introduction to numerical analysis, 2nd edn. Springer, New York Stoer J, Bulirsch R (1991) Introduction to numerical analysis, 2nd edn. Springer, New York
Zurück zum Zitat Tay ALP, Zurada JM, Wong LP, Xu J et al (2007) The hierarchical fast learning artificial neural network (HieFLANN)—an autonomous platform for hierarchical neural network construction. IEEE Trans Neural Netw 18(6):1645–1657CrossRef Tay ALP, Zurada JM, Wong LP, Xu J et al (2007) The hierarchical fast learning artificial neural network (HieFLANN)—an autonomous platform for hierarchical neural network construction. IEEE Trans Neural Netw 18(6):1645–1657CrossRef
Zurück zum Zitat Trefethen LN, Bau D III (1997) Numerical linear algebra. Society for Industrial and Applied Mathematics, Philadelphia. ISBN 978-0-89871-361-9 Trefethen LN, Bau D III (1997) Numerical linear algebra. Society for Industrial and Applied Mathematics, Philadelphia. ISBN 978-0-89871-361-9
Zurück zum Zitat Zurada J, Aizenberg I, Mazurowski M (2008) Learning in networks: complex-valued, pruning, and rule extraction. Proceedings of the 4th IEEE International Conference on Intelligent Systems (IS-2008), vol 1, Varna, Bulgaria, Sep 2008, pp 15–20 Zurada J, Aizenberg I, Mazurowski M (2008) Learning in networks: complex-valued, pruning, and rule extraction. Proceedings of the 4th IEEE International Conference on Intelligent Systems (IS-2008), vol 1, Varna, Bulgaria, Sep 2008, pp 15–20
Metadaten
Titel
A modified learning algorithm for the multilayer neural network with multi-valued neurons based on the complex QR decomposition
verfasst von
Igor Aizenberg
Antonio Luchetta
Stefano Manetti
Publikationsdatum
01.04.2012
Verlag
Springer-Verlag
Erschienen in
Soft Computing / Ausgabe 4/2012
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-011-0755-7

Weitere Artikel der Ausgabe 4/2012

Soft Computing 4/2012 Zur Ausgabe