Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Neural Computing and Applications
Erschienen in: Neural Computing and Applications 7/2020

29.09.2018 | Original Article

Convergence of a modified gradient-based learning algorithm with penalty for single-hidden-layer feed-forward networks

verfasst von: Jian Wang, Bingjie Zhang, Zhaoyang Sang, Yusong Liu, Shujun Wu, Quan Miao

Erschienen in: Neural Computing and Applications | Ausgabe 7/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Based on a novel algorithm, known as the upper-layer-solution-aware (USA), a new algorithm, in which the penalty method is introduced into the empirical risk, is studied for training feed-forward neural networks in this paper, named as USA with penalty. Both theoretical analysis and numerical results show that it can control the magnitude of weights of the networks. Moreover, the deterministic theoretical analysis of the new algorithm is proved. The monotonicity of the empirical risk with penalty term is guaranteed in the training procedure. The weak and strong convergence results indicate that the gradient of the total error function with respect to weights tends to zero, and the weight sequence goes to a fixed point when the iterations approach positive infinity. Numerical experiment has been implemented and effectively verifies the proved theoretical results.
Vorheriger Artikel Self-adaptive global mine blast algorithm for numerical optimization
Nächster Artikel River discharge simulation using variable parameter McCarthy–Muskingum and wavelet-support vector machine methods

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
1.
Bishop CM (1993) Neural networks for pattern recognition. MIT Press, CambridgeMATH
2.
Wan EA (1990) Neural network classification: a Bayesian interpretation. IEEE Trans Neural Netw 1(4):303–305MathSciNetCrossRef
3.
4.
Zhang K, Ma XP, Li YL, Wu HY, Cui CY, Zhang XM, Zhang H, Yao J (2018) Parameter prediction of hydraulic fracture for tight reservoir based on micro-seismic and history matching. Fractals 26(2):1–17
5.
Huang G Bin, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef
6.
7.
Leshno M, Lin VY, Pinkus A, Schocken S (1993) Multilayer feedforward networks with a nonpolynomial activation function can approximate any function. Neural Netw 6(6):861–867CrossRef
8.
Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257CrossRef
9.
Werbos PJ (1974) Beyond regression: new tools for prediction and analysis in the behavioral sciences. Dissertation, Harvard University
10.
Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by back-propagating errors. MIT Press, CambridgeCrossRef
11.
Goodband JH, Haas OC, Mills JA (2008) A comparison of neural network approaches for on-line prediction in IGRT. Med Phys 35(3):1113–1122CrossRef
12.
Huang G Bin, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: IEEE International joint conference on neural networks, pp 985–990
13.
Huang GB, Siew CK (2005) Extreme learning machine with randomly assigned RBF kernels. Int J Inf Technol 11(1):16–24
14.
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501CrossRef
15.
Li MB, Huang GB, Saratchandran P, Sundararajan N (2005) Fully complex extreme learning machine. Neurocomputing 68(1):306–314CrossRef
16.
Huang G Bin, Siew CK (2004) Extreme learning machine: RBF network case. In: Control, automation, robotics and vision conference, pp 1029–1036
17.
Zhu WT, Miao J, Qing L (2014) Constrained extreme learning machine: a novel highly discriminative random feedforward neural network. In: International joint conference on neural networks. IEEE, pp 800–807
18.
Zhu QY, Qin AK, Suganthan PN, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recognit 38(10):1759–1763CrossRef
19.
Ding S, Zhao H, Zhang Y, Xu X, Nie R (2015) Extreme learning machine: algorithm, theory and applications. Artif Intell Rev 44(1):103–115CrossRef
20.
Yu D, Deng L (2012) Efficient and effective algorithms for training single-hidden-layer neural networks. Pattern Recognit Lett 33(5):554–558MathSciNetCrossRef
21.
Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71(16–18):3460–3468CrossRef
22.
Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36(3):285–305CrossRef
23.
Huynh HT, Won Y, Kim JJ (2008) An improvement of extreme learning machine for compact single-hidden-layer feedforward neural networks. Int J Neural Syst 18(5):433–441CrossRef
24.
Han F, Yao HF, Ling QH (2013) An improved evolutionary extreme learning machine based on particle swarm optimization. Neurocomputing 116:87–93CrossRef
25.
Yu D, Tashev I (2014) Speech emotion recognition using deep neural network and extreme learning machine. In: Interspeech, pp 223–227
26.
Wang Y, Li D, Yi D, Zhisong P (2015) Anomaly detection in traffic using L1-norm minimization extreme learning machine. Neurocomputing 149:415–425CrossRef
27.
Hinton GE (1989) Connectionist learning procedures. Artif Intell 40(13):185–234CrossRef
28.
Reed R (1993) Pruning algorithm—a survey. IEEE Trans Neural Netw 4(5):740–747CrossRef
29.
Ishikawa M (1996) Structural learning with forgetting. Neural Netw 9(3):509–521CrossRef
30.
Setiono R (1997) A penalty-function approach for pruning feedforward neural networks. Neural Comput 9(1):185–204CrossRef
31.
Haykin S (1994) Neural networks: a comprehensive foundation. Macmillan, New YorkMATH
32.
Tibshirani R (1994) Regression shrinkage and selection via the lasso. J R Stat Soc 58:267–288MathSciNetMATH
33.
Hoerl AE (1962) Application of ridge analysis to regression problems. Chem Eng Prog 58:54–59
34.
Tychonoff AN (1963) Solution of incorrectly formulated problems and the regularization method. Sov Math 4:1035–1038MATH
35.
Takase H, Kita H, Hayashi T (2003) Effect of regularization term upon fault tolerant training. In: International joint conference on neural networks, pp 1048–1053
36.
Hoerl AE, Kennard RW (1970) Ridge regression: biased estimation for nonorthogonal problems. Technometrics 42(1):80–86CrossRef
37.
Sun W, Yuan YX (2001) Optimization theory and methods. Science Press, Beijing
38.
Metadaten
Titel
Convergence of a modified gradient-based learning algorithm with penalty for single-hidden-layer feed-forward networks
verfasst von
Jian Wang
Bingjie Zhang
Zhaoyang Sang
Yusong Liu
Shujun Wu
Quan Miao
Publikationsdatum
29.09.2018
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 7/2020
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-018-3748-y

Weitere Artikel der Ausgabe 7/2020

Neural Computing and Applications 7/2020 Zur Ausgabe

Original Article

Self-adaptive global mine blast algorithm for numerical optimization

Original Article

Integrating mutation scheme into monarch butterfly algorithm for global numerical optimization

Editorial

Special issue on deep learning and neural computing for intelligent sensing and control

Original Article

DC–DC converters design using a type-2 wavelet fuzzy cerebellar model articulation controller

Original Article

Distance and signal quality aware next hop selection routing protocol for vehicular ad hoc networks

Original Article

Auto-MeDiSine: an auto-tunable medical decision support engine using an automated class outlier detection method and AutoMLP

Premium Partner

    Bildnachweise