Skip to main content
Erschienen in: Neural Computing and Applications 6/2009

01.09.2009 | Original Article

Neural network training with optimal bounded ellipsoid algorithm

verfasst von: José de Jesús Rubio, Wen Yu, Andrés Ferreyra

Erschienen in: Neural Computing and Applications | Ausgabe 6/2009

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Compared to normal learning algorithms, for example backpropagation, the optimal bounded ellipsoid (OBE) algorithm has some better properties, such as faster convergence, since it has a similar structure as Kalman filter. OBE has some advantages over Kalman filter training, the noise is not required to be Guassian. In this paper OBE algorithm is applied in training the weights of the feedforward neural network for nonlinear system identification. Both hidden layers and output layers can be updated. From a dynamic system point of view, such training can be useful for all neural network applications requiring real-time updating of the weights. Two simulations give the effectiveness of the suggested algorithm.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Chowdhury FN (2003) A new approach to real-time training of dynamic neural networks. Int J Adapt Control Signal Process 31:509–521CrossRef Chowdhury FN (2003) A new approach to real-time training of dynamic neural networks. Int J Adapt Control Signal Process 31:509–521CrossRef
3.
Zurück zum Zitat Correa MV, Aguirre LA, Saldanha RR (2002) Using steady-state prior knowledge to constrain parameter estimates in nonlinear system identification. IEEE Trans Circuits Syst Part I 49(9):1376–1381CrossRef Correa MV, Aguirre LA, Saldanha RR (2002) Using steady-state prior knowledge to constrain parameter estimates in nonlinear system identification. IEEE Trans Circuits Syst Part I 49(9):1376–1381CrossRef
4.
Zurück zum Zitat Dickerson JA, Kosko B (1996) Fuzzy function approximation with ellipsoidal rules. IEEE Trans Syst Man Cybern 26:542–560CrossRef Dickerson JA, Kosko B (1996) Fuzzy function approximation with ellipsoidal rules. IEEE Trans Syst Man Cybern 26:542–560CrossRef
5.
Zurück zum Zitat Fogel E, Huang YF (1982) On the value of information in system identification: bounded noise case. Automatica 18(2):229–238MATHCrossRefMathSciNet Fogel E, Huang YF (1982) On the value of information in system identification: bounded noise case. Automatica 18(2):229–238MATHCrossRefMathSciNet
6.
Zurück zum Zitat Gazor S, Shahtalebi K (2002) A new NLMS algorithm for slow noise magnitude variation. IEEE Signal Process Lett 9(11):348–351CrossRef Gazor S, Shahtalebi K (2002) A new NLMS algorithm for slow noise magnitude variation. IEEE Signal Process Lett 9(11):348–351CrossRef
7.
Zurück zum Zitat El Ghaoui (2001) Robust filtering for discrete-time systems with bounded noise and parametric uncertainty. IEEE Trans Autom Control 46(7):1084–1089MATHCrossRef El Ghaoui (2001) Robust filtering for discrete-time systems with bounded noise and parametric uncertainty. IEEE Trans Autom Control 46(7):1084–1089MATHCrossRef
8.
Zurück zum Zitat Gollamudi S, Nagaraj S, Kapoor S, Huang YF (1996) Set membership state estimation with optimal bounding ellipsoids. In: Proceedings of international symposium on information theory and its applications, Victoria, BC, Canada, pp 262–265 Gollamudi S, Nagaraj S, Kapoor S, Huang YF (1996) Set membership state estimation with optimal bounding ellipsoids. In: Proceedings of international symposium on information theory and its applications, Victoria, BC, Canada, pp 262–265
9.
Zurück zum Zitat Goodwin GC, Sang Sin K (1984) Adaptive filtering prediction and control. Prentice-Hall, Englewood Cliffis Goodwin GC, Sang Sin K (1984) Adaptive filtering prediction and control. Prentice-Hall, Englewood Cliffis
10.
Zurück zum Zitat Haykin S (1994) Neural networks—a comprehensive foundation. Macmillan, New YorkMATH Haykin S (1994) Neural networks—a comprehensive foundation. Macmillan, New YorkMATH
12.
Zurück zum Zitat Jin L, Nikiforunk PN, Gupta MM (1999) Adaptive model reference control of discrete-time nonlinear systems using neural networks. Control Theory Adv Technol Part 3 10(4):1373–1399 Jin L, Nikiforunk PN, Gupta MM (1999) Adaptive model reference control of discrete-time nonlinear systems using neural networks. Control Theory Adv Technol Part 3 10(4):1373–1399
13.
Zurück zum Zitat Joachim D, Deller JR (2006) Multiweight optimization in optimal bounding ellipsoid algorithms. IEEE Trans Signal Process 54(2):679–690CrossRef Joachim D, Deller JR (2006) Multiweight optimization in optimal bounding ellipsoid algorithms. IEEE Trans Signal Process 54(2):679–690CrossRef
14.
Zurück zum Zitat Kapoor S, Gollamudi S, Nagaraj S, Huang YF (1996) Tracking of time-varing parameters using optimal bounding ellipsoid algorithms. In: Proceedings of 34th Allerton conference on communication, control and computing Monticello Kapoor S, Gollamudi S, Nagaraj S, Huang YF (1996) Tracking of time-varing parameters using optimal bounding ellipsoid algorithms. In: Proceedings of 34th Allerton conference on communication, control and computing Monticello
15.
Zurück zum Zitat Kayuri NS, Vienkatasubramanian V (1993) Representing bounded fault classes using neural networks with ellipsoidal activation functions. Comput Chem Eng 17(2):139–163CrossRef Kayuri NS, Vienkatasubramanian V (1993) Representing bounded fault classes using neural networks with ellipsoidal activation functions. Comput Chem Eng 17(2):139–163CrossRef
16.
Zurück zum Zitat Kosmatopoulos EB, Polycarpou MM, Christodoulou MA, Ioannou PA (1995) High-order neural network structures for identification of dynamical systems. IEEE Trans Neural Netw 6(2):422–431CrossRef Kosmatopoulos EB, Polycarpou MM, Christodoulou MA, Ioannou PA (1995) High-order neural network structures for identification of dynamical systems. IEEE Trans Neural Netw 6(2):422–431CrossRef
17.
Zurück zum Zitat Liguni Y, Sakai H, Higekatsu Tokumaru (1992) A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter. IEEE Trans Signal Process 40(4):959–966CrossRef Liguni Y, Sakai H, Higekatsu Tokumaru (1992) A real-time learning algorithm for a multilayered neural network based on the extended Kalman filter. IEEE Trans Signal Process 40(4):959–966CrossRef
18.
19.
Zurück zum Zitat Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27CrossRef Narendra KS, Parthasarathy K (1990) Identification and control of dynamical systems using neural networks. IEEE Trans Neural Netw 1(1):4–27CrossRef
20.
Zurück zum Zitat Nazin SA, Polyak BT (2001) Limiting behavior of bounding ellipsoids for state estimation. In: Proceedings of the 5th IFAC symposium on nonlinear control systems, St Petersburg, Russia, vol 4–6, pp 585–589 Nazin SA, Polyak BT (2001) Limiting behavior of bounding ellipsoids for state estimation. In: Proceedings of the 5th IFAC symposium on nonlinear control systems, St Petersburg, Russia, vol 4–6, pp 585–589
21.
Zurück zum Zitat Parlos AG, Menon SK, Atiya AF (2001) An algorithm approach to adaptive state filtering using recurrent neural network. IEEE Trans Neural Netw 12(6):1411–1432CrossRef Parlos AG, Menon SK, Atiya AF (2001) An algorithm approach to adaptive state filtering using recurrent neural network. IEEE Trans Neural Netw 12(6):1411–1432CrossRef
22.
Zurück zum Zitat Puskorius GV, Feldkamp LA (1994) Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks. IEEE Trans Neural Netw 5(2):279–297CrossRef Puskorius GV, Feldkamp LA (1994) Neurocontrol of nonlinear dynamical systems with Kalman filter trained recurrent networks. IEEE Trans Neural Netw 5(2):279–297CrossRef
23.
Zurück zum Zitat Ros L, Sabater A, Thomas F (2002) An ellipsoidal alculus based on propagation and fusion. IEEE Trans Syst Man Cybern 32(4):430-442CrossRef Ros L, Sabater A, Thomas F (2002) An ellipsoidal alculus based on propagation and fusion. IEEE Trans Syst Man Cybern 32(4):430-442CrossRef
24.
Zurück zum Zitat Rubio JJ, Yu W (2005) Dead-zone Kalman filter algorithm for recurrent neural networks. In: 44rd IEEE conference on decision and control, CDC’05, Seville, Spain, pp 2562–2567 Rubio JJ, Yu W (2005) Dead-zone Kalman filter algorithm for recurrent neural networks. In: 44rd IEEE conference on decision and control, CDC’05, Seville, Spain, pp 2562–2567
25.
Zurück zum Zitat Ruck DW, Rogers SK, Kabrisky M, Maybeck PS, Oxley ME (1992) Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons. IEEE Trans Pattern Anal Mach Intell 14(6):686–691CrossRef Ruck DW, Rogers SK, Kabrisky M, Maybeck PS, Oxley ME (1992) Comparative analysis of backpropagation and the extended Kalman filter for training multilayer perceptrons. IEEE Trans Pattern Anal Mach Intell 14(6):686–691CrossRef
26.
Zurück zum Zitat Safonov MG, Athans M (1978) Robustness and computational aspects of nonlinear stochastic estimators and regulators. IEEE Trans Autom Control 23(4):717–725MATHCrossRefMathSciNet Safonov MG, Athans M (1978) Robustness and computational aspects of nonlinear stochastic estimators and regulators. IEEE Trans Autom Control 23(4):717–725MATHCrossRefMathSciNet
27.
Zurück zum Zitat Sastry PS, Santharam G, Unnikrishnan KP (1994) Memory neural networks for identification and control of dynamic systems. IEEE Trans Neural Netw 5:306–319CrossRef Sastry PS, Santharam G, Unnikrishnan KP (1994) Memory neural networks for identification and control of dynamic systems. IEEE Trans Neural Netw 5:306–319CrossRef
28.
Zurück zum Zitat Schueppe FC (1973) Uncertain dynamic systems. Prentice-Hall, Englewood Cliffs Schueppe FC (1973) Uncertain dynamic systems. Prentice-Hall, Englewood Cliffs
29.
Zurück zum Zitat Singhal S, Wu L (1989) Training multilayer perceptrons with the extended Kalman algorithm. Adv Neural Inf Process Syst I 133–140 Singhal S, Wu L (1989) Training multilayer perceptrons with the extended Kalman algorithm. Adv Neural Inf Process Syst I 133–140
30.
Zurück zum Zitat Weyer E, Campi MC (2000) Non-asymptotic confidence ellipsoids for the least squares estimate. In: 39rd IEEE conference on decision and control, Sydney, Australia, pp 2688–2693 Weyer E, Campi MC (2000) Non-asymptotic confidence ellipsoids for the least squares estimate. In: 39rd IEEE conference on decision and control, Sydney, Australia, pp 2688–2693
31.
Zurück zum Zitat Wang LX (1997) A course in fuzzy systems and control. Prentice-Hall, Englewood Cliffs Wang LX (1997) A course in fuzzy systems and control. Prentice-Hall, Englewood Cliffs
32.
Zurück zum Zitat Yu W (2002) Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms. Inf Sci 158(1):131–147 Yu W (2002) Nonlinear system identification using discrete-time recurrent neural networks with stable learning algorithms. Inf Sci 158(1):131–147
33.
Zurück zum Zitat Yu W, Ferreyra A (2005) On-line clustering for nonlinear system identification using fuzzy neural networks. In: IEEE international conference on fuzzy systems, pp 678–683 Yu W, Ferreyra A (2005) On-line clustering for nonlinear system identification using fuzzy neural networks. In: IEEE international conference on fuzzy systems, pp 678–683
Metadaten
Titel
Neural network training with optimal bounded ellipsoid algorithm
verfasst von
José de Jesús Rubio
Wen Yu
Andrés Ferreyra
Publikationsdatum
01.09.2009
Verlag
Springer-Verlag
Erschienen in
Neural Computing and Applications / Ausgabe 6/2009
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-008-0203-5

Weitere Artikel der Ausgabe 6/2009

Neural Computing and Applications 6/2009 Zur Ausgabe

Premium Partner