Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
International Journal of Machine Learning and Cybernetics
Erschienen in: International Journal of Machine Learning and Cybernetics 12/2023

12.07.2023 | Original Article

Convergence analysis for sparse Pi-sigma neural network model with entropy error function

verfasst von: Qinwei Fan, Fengjiao Zheng, Xiaodi Huang, Dongpo Xu

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 12/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

As a high-order neural network, the Pi-sigma neural network has demonstrated its capacities for fast learning and strong nonlinear processing. In this paper, a new algorithm is proposed for Pi-sigma neural networks with entropy error functions based on \(L_{0}\) regularization. One of the key features of the proposed algorithm is the use of an entropy error function instead of the more common square error function, which is different from those in most existing literature. At the same time, the proposed algorithm also employs \(L_{0}\) regularization as a means of ensuring the efficiency of the network. Based on the gradient method, the monotonicity, and strong and weak convergence of the network are strictly proved by theoretical analysis and experimental verification. Experiments on applying the proposed algorithm to both classification and regression problems have demonstrated the improved performance of the algorithm.
Vorheriger Artikel General fine-grained event detection based on fusion of multi-information representation and attention mechanism
Nächster Artikel Efficient dual domain image denoising via SURE-based optimization

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Weitere Produktempfehlungen anzeigen
Literatur
1.
Shin Y, Ghosh J (1991) The pi-sigma network: an efficient higher-order neural network for pattern classification and function approximation. IEEE 1:13–18
2.
Kang X, Yan X, Chao Z et al (2007) Convergence of online gradient algorithm with stochastic inputs for pi-sigma neural networks. IEEE
3.
4.
De Ridder D, Duin RPW, Egmont-Petersen M et al (2003) Nonlinear image processing using artificial neural networks. Elsevier 126:351–450
5.
Patel JL, Goyal RK (2007) Applications of artificial neural networks in medical science. Curr Clin Pharmacol 2(3):217–226CrossRef
6.
Hussain AJ, Liatsis P (2003) Recurrent pi-sigma networks for DPCM image coding. Neurocomputing 55(1–2):363–382CrossRef
7.
Jiang LJ (2005) Application of Pi-Sigma neural network to real-time classification of seafloor sediments. Appl Acoust 20:20
8.
Wang F, Wang Y, Tian Y et al (2019) Pattern recognition and prognostic analysis of longitudinal blood pressure records in hemodialysis treatment based on a convolutional neural network[J]. J Biomed Inform 98:103271CrossRef
9.
Babic M, Marina N, Mrvar A et al (2019) A new method for biostatistical miRNA pattern recognition with topological properties of visibility graphs in 3D space. J Healthc Eng 20:20
10.
Fan Q, Peng J, Li H, Lin S (2021) Convergence of a gradient-based learning algorithm with penalty for ridge polynomial neural networks. IEEE Access 9:28742–28752CrossRef
11.
Wu W, Xu Y (2002) Deterministic convergence of an online gradient method for neural networks. J Comput Appl Math 144(1–2):335–347MathSciNetMATHCrossRef
12.
Liu Y, Yang J, Yang D et al (2014) A modified gradient based neuro fuzzy learning algorithm for Pi-Sigma network based on first order takagi sugeno system. J Math Res Appl 34(1):114–126MathSciNetMATH
13.
Mohamed KS, Wu W, Liu Y (2017) A modified higher-order feed forward neural network with smoothing regularization. Neural Netw World 27(6):577–592CrossRef
14.
Kang Q, Fan Q, Zurada JM (2021) Deterministic convergence analysis via smoothing group Lasso regularization and adaptive momentum for sigma-pi-sigma neural network. Inf Sci 553:66–82MathSciNetMATHCrossRef
15.
Fan Q, Kang Q, Zurada JM (2022) Convergence analysis for sigma-pi-sigma neural network based on some relaxed conditions. Inf Sci 585:70–88CrossRef
16.
Falas T, Stafylopatis AG (1999) The impact of the error function selection in neural network-based classifiers. IEEE 3:1799–1804
17.
Li L, Qiao Z, Long Z (2020) A smoothing algorithm with constant learning rate for training two kinds of fuzzy neural networks and its convergence. Neural Process Lett 51:1093–1109CrossRef
18.
Huang C, Liu B, Tian X et al (2019) Global convergence on asymptotically almost periodic SICNNs with nonlinear decay functions. Neural Process Lett 49:625–641CrossRef
19.
Xu D, Dong J, Zhang H (2017) Deterministic convergence of Wirtinger-gradient methods for complex-valued neural networks. Neural Process Lett 45:445–456CrossRef
20.
Song D, Zhang Y, Shan X et al (2017) Over-learning phenomenon of wavelet neural networks in remote sensing image classifications with different entropy error functions. Entropy 19(3):101CrossRef
21.
Karayiannis NB, Venetsanopoulos AN, Karayiannis NB et al (1993) Fast learning algorithms for neural networks. Artif Neural Netw Learn Algorithms Perform Eval Appl 20:141–193MATH
22.
Oh SH (1997) Improving the error backpropagation algorithm with a modified error function. IEEE Trans Neural Netw 8(3):799–803CrossRef
23.
Xiong Y, Tong X (2020) Convergence of batch gradient method based on the entropy error function for feedforward neural networks. Neural Process Lett 52(3):2687–2695CrossRef
24.
Lin KWE, Balamurali BT, Koh E et al (2020) Singing voice separation using a deep convolutional neural network trained by ideal binary mask and cross entropy. Neural Comput Appl 32(4):1037–1050CrossRef
25.
Shan B, Fang Y (2020) A cross entropy based deep neural network model for road extraction from satellite images. Entropy 22(5):535MathSciNetCrossRef
26.
Bahri A, Majelan SG, Mohammadi S et al (2019) Remote sensing image classification via improved cross-entropy loss and transfer learning strategy based on deep convolutional neural networks. IEEE Geosci Remote Sens Lett 17(6):1087–1091CrossRef
27.
Wang Y, Chen X, Dong K (2019) Attribute reduction via local conditional entropy. Int J Mach Learn Cybern 10:3619–3634CrossRef
28.
Bosman AS, Engelbrecht A, Helbig M (2020) Visualising basins of attraction for the cross-entropy and the squared error neural network loss functions. Neurocomputing 400:113–136CrossRef
29.
Martin R (2005) Speech enhancement based on minimum mean-square error estimation and supergaussian priors. IEEE Trans Speech Audio Process 13(5):845–856CrossRef
30.
Zhang H, Jiang Y, Wang J et al (2022) Bilateral sensitivity analysis: a better understanding of a neural network and its application to reservoir engineering. Int J Mach Learn Cybern 13(8):2135–2152CrossRef
31.
Liu X, Dai J, Chen J et al (2020) Unsupervised attribute reduction based on \({\alpha}\)-approximate equal relation in interval-valued information systems. Int J Mach Learn Cybern 11(9):2021–2038CrossRef
32.
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press, New YorkMATH
33.
34.
Ma L, Bian W (2021) A simple neural network for sparse optimization with \(L_{1}\) regularization. IEEE Trans Netw Sci Eng 8(4):3430–3442CrossRef
35.
Liang Y, Liu C, Luan XZ et al (2013) Sparse logistic regression with a \(L_{1/2}\) penalty for gene selection in cancer classification. BMC Bioinform 14(1):1–12CrossRef
36.
Khan A, Yang J, Wu W (2014) Double parallel feedforward neural network based on extreme learning machine with \(L_{1/2}\) regularizer. Neurocomputing 128:113–118CrossRef
37.
Wang Y, Liu P, Li Z et al (2013) Data regularization using Gaussian beams decomposition and sparse norms. J Inverse Ill-Posed Probl 21(1):1–23MathSciNetMATHCrossRef
38.
Louizos C, Welling M, Kingma DP (2017) Learning sparse neural networks through \(L_{0}\) regularization. xarXiv:​1712.​01312 (arXiv preprint)
39.
Woeginger GJ (2003) Exact algorithms for NP-hard problems: a survey. Springer, Berlin, pp 185–207MATH
40.
Fan Q, Zurada JM, Wu W (2014) Convergence of online gradient method for feedforward neural networks with smoothing \(L_{1/2}\) regularization penalty. Neurocomputing 131:208–216CrossRef
41.
Wu W, Fan Q, Zurada JM et al (2014) Batch gradient method with smoothing \(L_{1/2}\) regularization for training of feedforward neural networks. Neural Netw 50:72–78MATHCrossRef
42.
Liu Y, Yang D, Zhang C (2018) Relaxed conditions for convergence analysis of online back-propagation algorithm with \(L_{2}\) regularizer for Sigma-Pi-Sigma neural network. Neurocomputing 272:163–169CrossRef
43.
Xie X, Zhang H, Wang J et al (2019) Learning optimized structure of neural networks by hidden node pruning with \(L_{1}\) regularization. IEEE Trans Cybern 50(3):1333–1346CrossRef
44.
Zhang H, Wang J, Wang J et al (2020) Feature selection using a neural network with group lasso regularization and controlled redundancy. IEEE Trans Neural Netw Learn Syst 32(3):1110–1123
45.
Sun W, Yuan YX (2006) Optimization theory and methods: nonlinear programming. Springer, BerlinMATH
Metadaten
Titel
Convergence analysis for sparse Pi-sigma neural network model with entropy error function
verfasst von
Qinwei Fan
Fengjiao Zheng
Xiaodi Huang
Dongpo Xu
Publikationsdatum
12.07.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 12/2023
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-023-01901-x

Weitere Artikel der Ausgabe 12/2023

International Journal of Machine Learning and Cybernetics 12/2023 Zur Ausgabe

Original Article

A novel stratification clustering algorithm based on a new local density estimation method and an improved local inter-cluster distance measure

Original Article

A causality-inspired data augmentation approach to cross-domain burr detection using randomly weighted shallow networks

Original Article

Multiple sparse spaces network pruning via a joint similarity criterion

Original Article

Multi-scale fusion transformer based weakly supervised hashing learning for instance retrieval

Original Article

Unsupervised concept drift detection method based on robust random cut forest

Original Article

Efficient dual domain image denoising via SURE-based optimization

Neuer Inhalt

    Bildnachweise