Skip to main content
Erschienen in: Cognitive Neurodynamics 3/2014

01.06.2014 | Brief Communication

Convergence analysis of fully complex backpropagation algorithm based on Wirtinger calculus

verfasst von: Huisheng Zhang, Xiaodong Liu, Dongpo Xu, Ying Zhang

Erschienen in: Cognitive Neurodynamics | Ausgabe 3/2014

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper considers the fully complex backpropagation algorithm (FCBPA) for training the fully complex-valued neural networks. We prove both the weak convergence and strong convergence of FCBPA under mild conditions. The decreasing monotonicity of the error functions during the training process is also obtained. The derivation and analysis of the algorithm are under the framework of Wirtinger calculus, which greatly reduces the description complexity. The theoretical results are substantiated by a simulation example.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Adali T, Li H, Novey M et al (2008) Complex ICA using nonlinear functions. IEEE Trans Signal Process 56(9):4356–544 Adali T, Li H, Novey M et al (2008) Complex ICA using nonlinear functions. IEEE Trans Signal Process 56(9):4356–544
Zurück zum Zitat Bos AVD (1994) Complex gradient and Hessian. Proc Inst Electr Eng Vision Image Signal Process 141:380–382CrossRef Bos AVD (1994) Complex gradient and Hessian. Proc Inst Electr Eng Vision Image Signal Process 141:380–382CrossRef
Zurück zum Zitat Brandwood D (1983) Complex gradient operator and its application in adaptive array theory. Proc Inst Electr Eng 130:11–16 Brandwood D (1983) Complex gradient operator and its application in adaptive array theory. Proc Inst Electr Eng 130:11–16
Zurück zum Zitat Fink O, Zio E, Weidmann U (2014) Predicting component reliability and level of degradation with complex-valued neural networks. Reliab Eng Syst Safe 121:198–206CrossRef Fink O, Zio E, Weidmann U (2014) Predicting component reliability and level of degradation with complex-valued neural networks. Reliab Eng Syst Safe 121:198–206CrossRef
Zurück zum Zitat Hirose A (2012) Complex-valued neural networks. Springer-Verlag, Berlin HeidelbergCrossRef Hirose A (2012) Complex-valued neural networks. Springer-Verlag, Berlin HeidelbergCrossRef
Zurück zum Zitat Kim T, Adali T (2003) Approximation by fully complex multilayer perceptrons. Neural Comput 15:1641–666PubMedCrossRef Kim T, Adali T (2003) Approximation by fully complex multilayer perceptrons. Neural Comput 15:1641–666PubMedCrossRef
Zurück zum Zitat Li H, Adali T (2008) Complex-valued adaptive signal processing using nonlinear functions. EURASIP J Adv Signal Process 2008:122 Li H, Adali T (2008) Complex-valued adaptive signal processing using nonlinear functions. EURASIP J Adv Signal Process 2008:122
Zurück zum Zitat Mandi DP, Goh SL (2009) Complex valued nonlinear adaptive filters. Wiley, ChichesterCrossRef Mandi DP, Goh SL (2009) Complex valued nonlinear adaptive filters. Wiley, ChichesterCrossRef
Zurück zum Zitat Mcleod RM (1965) Mean value theorems for vector valued functions. Proc Edinburgh Math Soc 14(2):197–209CrossRef Mcleod RM (1965) Mean value theorems for vector valued functions. Proc Edinburgh Math Soc 14(2):197–209CrossRef
Zurück zum Zitat Nitta T (1997) An extension of the back-propagation algorithm to complex numbers. Neural Netw 10(8): 1391–1415PubMedCrossRef Nitta T (1997) An extension of the back-propagation algorithm to complex numbers. Neural Netw 10(8): 1391–1415PubMedCrossRef
Zurück zum Zitat Nitta T (2013) Local minima in hierarchical structures of complex-valued neural networks. Neural Netw 43: 1–7PubMedCrossRef Nitta T (2013) Local minima in hierarchical structures of complex-valued neural networks. Neural Netw 43: 1–7PubMedCrossRef
Zurück zum Zitat Osborn GW (2010) A Kalman filtering approach to the representation of kinematic quantities by the hippocampal-entorhinal complex. Cogn Neurodyn 4:C315–C335CrossRef Osborn GW (2010) A Kalman filtering approach to the representation of kinematic quantities by the hippocampal-entorhinal complex. Cogn Neurodyn 4:C315–C335CrossRef
Zurück zum Zitat Shao HM, Zheng GF (2011) Boundedness and convergence of online gradient method with penalty and momentum. Neurocomputing 74:765–770CrossRef Shao HM, Zheng GF (2011) Boundedness and convergence of online gradient method with penalty and momentum. Neurocomputing 74:765–770CrossRef
Zurück zum Zitat Wei H, Ren Y, Wang ZY (2013) A computational neural model of orientation detection based on multiple guesses: comparison of geometrical and algebraic models. Cogn Neurodyn 7:C361–C379CrossRef Wei H, Ren Y, Wang ZY (2013) A computational neural model of orientation detection based on multiple guesses: comparison of geometrical and algebraic models. Cogn Neurodyn 7:C361–C379CrossRef
Zurück zum Zitat Wu W, Feng GR, Li ZX, et al. (2005) Deterministic convergence of an online gradient method for BP neural networks. IEEE Trans Neural Netw 16:533–540PubMedCrossRef Wu W, Feng GR, Li ZX, et al. (2005) Deterministic convergence of an online gradient method for BP neural networks. IEEE Trans Neural Netw 16:533–540PubMedCrossRef
Zurück zum Zitat Wang J, Wu W, Zurada J (2011) Deterministic convergence of conjugate gradient method for feedforward neural networks. Neurocomputing 74:2368–2376CrossRef Wang J, Wu W, Zurada J (2011) Deterministic convergence of conjugate gradient method for feedforward neural networks. Neurocomputing 74:2368–2376CrossRef
Zurück zum Zitat Xu DP, Zhang HS, Liu L (2010) Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural networks. Neural Comput 22(10):2655-2677PubMedCrossRef Xu DP, Zhang HS, Liu L (2010) Convergence analysis of three classes of split-complex gradient algorithms for complex-valued recurrent neural networks. Neural Comput 22(10):2655-2677PubMedCrossRef
Zurück zum Zitat Zhang C, Wu W, Xiong Y (2007) Convergence analysis of batch gradient algorithm for three classes of sigma-pi neural networks. Neural Process Lett 26:177–180CrossRef Zhang C, Wu W, Xiong Y (2007) Convergence analysis of batch gradient algorithm for three classes of sigma-pi neural networks. Neural Process Lett 26:177–180CrossRef
Zurück zum Zitat Zhang C, Wu W, Chen XH, et al. (2008) Convergence of BP algorithm for product unit neural networks with exponential weights. Neurocomputing 72:513–520CrossRef Zhang C, Wu W, Chen XH, et al. (2008) Convergence of BP algorithm for product unit neural networks with exponential weights. Neurocomputing 72:513–520CrossRef
Zurück zum Zitat Zhang HS, Wu W, Liu F, Yao MC (2009) Boundedness and convergence of online gradient method with penalty for feedforward neural networks. IEEE Trans Neural Netw 20(6):1050–1054PubMedCrossRef Zhang HS, Wu W, Liu F, Yao MC (2009) Boundedness and convergence of online gradient method with penalty for feedforward neural networks. IEEE Trans Neural Netw 20(6):1050–1054PubMedCrossRef
Zurück zum Zitat Zhang HS, Xu DP, Zhang Y (2013) Boundedness and convergence of split-complex back-propagation algorithm with momentum and penalty. Neural Process Lett. doi:10.1007/s11063-013-9305-x Zhang HS, Xu DP, Zhang Y (2013) Boundedness and convergence of split-complex back-propagation algorithm with momentum and penalty. Neural Process Lett. doi:10.​1007/​s11063-013-9305-x
Zurück zum Zitat Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables. Academic Press, New York Ortega JM, Rheinboldt WC (1970) Iterative solution of nonlinear equations in several variables. Academic Press, New York
Metadaten
Titel
Convergence analysis of fully complex backpropagation algorithm based on Wirtinger calculus
verfasst von
Huisheng Zhang
Xiaodong Liu
Dongpo Xu
Ying Zhang
Publikationsdatum
01.06.2014
Verlag
Springer Netherlands
Erschienen in
Cognitive Neurodynamics / Ausgabe 3/2014
Print ISSN: 1871-4080
Elektronische ISSN: 1871-4099
DOI
https://doi.org/10.1007/s11571-013-9276-7

Weitere Artikel der Ausgabe 3/2014

Cognitive Neurodynamics 3/2014 Zur Ausgabe

Neuer Inhalt