Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 9/2022

06-04-2022 | Original Article

Convergence analysis on the deterministic mini-batch learning algorithm for noise resilient radial basis function networks

Published in: International Journal of Machine Learning and Cybernetics | Issue 9/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper gives a formal convergence analysis on the mini-batch training algorithm for noise resilient radial basis function (RBF) networks. Unlike the conventional analysis which assumes that the mini-batch process is operated in a stochastic manner, we consider that the mini-batch training process is operated in a deterministic manner. The deterministic process divides the training samples into a number of fixed mini-batches, and the mini-batches are presented in a fixed order. This paper first states the noise resilient objective function for weight noise and weight fault. We then derive the mini-batch training algorithm for this noise resilient objective function. Our main contribution is the convergence analysis on the mini-batch training algorithm. We show that under the deterministic setting, the mini-batch training algorithm converges. The converged weight vector is asymptotically close to the optimal batch mode solution. Also, we derive the sufficient conditions (the learning rate range) for convergence. Our theoretical results can be applied to not only the noise resilient objective function but also a large class of objective functions.
previous article A sequential attention interface with a dense reward function for mitosis detection
next article Restricted subgradient descend method for sparse signal learning

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Footnotes
1
In this case, the numbers \(\kappa _i\)’s of samples in mini-batches should be greater than the number M of RBF nodes.
 
2
Note that to the best of knowledge, there is not other mini-batch algorithms for the noise resilient issue.
 
Literature
1.
Movassagh AA, Alzubi JA, Gheisari M, Rahimi M, Mohan S, Abbasi AA, Nabipour N (2021) Artificial neural networks training algorithm integrating invasive weed optimization with differential evolutionary model, J Ambient Intell Human Comput 1–9
2.
Soni B, Mathur P, Bora A (2021) In depth analysis, applications and future issues of artificial neural network. In: Enabling AI applications in data science, Springer, pp 149–183
3.
Mhara MAOA (2021) Complexity neural networks for estimating flood process in internet-of-things empowered smart city, Available at SSRN 3775433
4.
Gheisari M, Najafabadi HE, Alzubi JA, Gao J, Wang G, Abbasi AA, Castiglione A (2021) Obpp: An ontology-based framework for privacy-preserving in iot-based smart city. Fut Generation Comput Syst 123:1–13CrossRef
5.
Chandrasekaran K, Selvaraj J, Amaladoss CR, Veerapan L (2021) Hybrid renewable energy based smart grid system for reactive power management and voltage profile enhancement using artificial neural network, Energy Sources, Part A: Recovery, Utilization, and Environmental Effects, 1–24
6.
Alzubi OA, Nazir J, Hamdoun H (2015) Cyber attack challenges and resilience for smart grids, Euro J Sci Res
7.
Abukharis S, Alzubi JA, Alzubi OA, Alamri S (2014) Packet error rate performance of ieee802. 11g under bluetooth interface. Res J Appl Sci Eng Technol 8(12):1419–1423CrossRef
8.
Chan Y-C, Wong EW, Leung CS (2021) Evaluating non-hierarchical overflow loss systems using teletraffic theory and neural networks. IEEE Commun Lett 25(5):1486–1490CrossRef
9.
Alzubi JA (2015) Optimal classifier ensemble design based on cooperative game theory. Res J Appl Sci Eng Technol 11(12):1336–1343CrossRef
10.
Rzepecki Ł, Jaśkowski P (2021) Application of game theory against nature in supporting bid pricing in construction. Symmetry 13(1):132CrossRef
11.
Huang G, Huang G-B, Song S, You K (2015) Trends in extreme learning machines: a review. Neural Netw 61:32–48MATHCrossRef
12.
Zhang J, Li Y, Xiao W, Zhang Z (2020) Non-iterative and fast deep learning: multilayer extreme learning machines. J Frankl Inst 357(13):8925–8955MathSciNetMATHCrossRef
13.
Zhang J, Xiao W, Li Y, Zhang S (2018) Residual compensation extreme learning machine for regression. Neurocomputing 311:126–136CrossRef
14.
Leung C-S, Wan WY, Feng R (2016) A regularizer approach for RBF networks under the concurrent weight failure situation. IEEE Trans Neural Netw Learn Syst 28(6):1360–1372CrossRef
15.
Haykin S (1999) Neural networks: a comprehensive foundation, 2nd edn. Prentice Hall, Upper Saddle River, NJ, USAMATH
16.
Karamichailidou D, Kaloutsa V, Alexandridis A (2021) Wind turbine power curve modeling using radial basis function neural networks and tabu search. Renewable Energy 163:2137–2152CrossRef
17.
Wu H, Han Y, Geng Z, Fan J, Xu W (2022) Production capacity assessment and carbon reduction of industrial processes based on novel radial basis function integrating multi-dimensional scaling. Sustain Energy Technol Assess 49
18.
Fei J, Wang T (2019) Adaptive fuzzy-neural-network based on rbfnn control for active power filter. Int J Mach Learn Cybernet 10(5):1139–1150CrossRef
19.
20.
Jin X, Sun W, Jin Z (2020) A discriminative deep association learning for facial expression recognition. Int J Mach Learn Cybernet 11(4):779–793CrossRef
21.
Cheng E-J, Chou K-P, Rajora S, Jin B-H, Tanveer M, Lin C-T, Young K-Y, Lin W-C, Prasad M (2019) Deep sparse representation classifier for facial recognition and detection system. Pattern Recognit Lett 125:71–77CrossRef
22.
Ghosh S, Pal A, Jaiswal S, Santosh K, Das N, Nasipuri M (2019) Segfast-v2: Semantic image segmentation with less parameters in deep learning for autonomous driving. Int J Mach Learn Cybernet 10(11):3145–3154CrossRef
23.
Fujiyoshi H, Hirakawa T, Yamashita T (2019) Deep learning-based image recognition for autonomous driving. IATSS Res 43(4):244–252CrossRef
24.
Wang Z, Zhou X, Wang W, Liang C (2020) Emotion recognition using multimodal deep learning in multiple psychophysiological signals and video. Int J Mach Learn Cybernet 11(4):923–934CrossRef
25.
Si C, Tao Y, Qiu J, Lai S, Zhao J (2021) Deep reinforcement learning based home energy management system with devices operational dependencies. Int J Mach Learn Cybernet 12(6):1687–1703CrossRef
26.
Zhou Y, Wang J, Liu Y, Yan R, Ma Y (2021) Incorporating deep learning of load predictions to enhance the optimal active energy management of combined cooling, heating and power system, Energy, 121134
27.
Wang X, Zhao Y, Pourpanah F (2020) Recent advances in deep learning
28.
Voulodimos A, Doulamis N, Doulamis A, Protopapadakis E (2018) Deep learning for computer vision: a brief review, Comput Intell Neurosci 2018
29.
Torres JM, Comesaña CI, Garcia-Nieto PJ (2019) Machine learning techniques applied to cybersecurity. Int J Mach Learn Cybernet 10(10):2823–2836CrossRef
30.
Ni D, Xiao Z, Lim MK (2019) A systematic review of the research trends of machine learning in supply chain management, Int J Mach Learn Cybernet, 1–20
31.
Sum J, Leung C-S, Ho K (2012) Convergence analyses on on-line weight noise injection-based training algorithms for MLPs. IEEE Trans Neural Netw Learn Syst 23(11):1827–1840CrossRef
32.
Zhang H, Wu W, Liu F, Yao M (2009) Boundedness and convergence of online gradient method with penalty for feedforward neural networks. IEEE Trans Neural Netw 20(6):1050–1054CrossRef
33.
White H (1989) Some asymptotic results for learning in single hidden-layer feedforward network models. J Am Stat Assoc 84(408):1003–1013MathSciNetMATHCrossRef
34.
Liu B, Kaneko T (1969) Error analysis of digital filters realized with floating-point arithmetic. Proc IEEE 57(10):1735–1747CrossRef
35.
Jeannerod C-P, Rump SM (2013) Improved error bounds for inner products in floating-point arithmetic. SIAM J Matrix Anal Appl 34(2):338–344MathSciNetMATHCrossRef
36.
Diniz PS (2020) The least-mean-square (lms) algorithm, In: Adaptive Filtering, Springer, pp 61–102
37.
Burr JB (1991) Digital neural network implementations. Neural Netw Concepts Appl Implementations 3:237–285
38.
Bolt G, Austin J, Morgan G (1992) Fault tolerant multi-layer perceptron networks, Citeseer
39.
Martolia R, Jain A, Singla L (2015) Analysis & survey on fault tolerance in radial basis function networks, In: International Conference on Computing, Communication & Automation, IEEE, pp 469–473
40.
Murakami M, Honda N (2007) Fault tolerance comparison of ids models with multilayer perceptron and radial basis function networks, In: 2007 International Joint Conference on Neural Networks, IEEE, pp 1079–1084
41.
Liu S-M, Tang L, Huang N-C, Tsai D-Y, Yang M-X, Wu K-C (2020) Fault-tolerance mechanism analysis on nvdla-based design using open neural network compiler and quantization calibrator, In: 2020 International symposium on VLSI design, automation and test (VLSI-DAT), IEEE, pp 1–3
42.
Yamazaki K, Tsutsumi T, Takahashi H, Higami Y, Aikyo T, Takamatsu Y, Yotsuyanagi H, Hashizume M (2009) A novel approach for improving the quality of open fault diagnosis, In: 2009 22nd International Conference on VLSI Design, IEEE, pp 85–90
43.
Leung CS, Wang H-J, Sum J (2010) On the selection of weight decay parameter for faulty networks. IEEE Trans Neural Netw 21(8):1232–1244CrossRef
44.
Leung C-S, Sum JP-F (2012) RBF networks under the concurrent fault situation. IEEE Trans Neural Netw Learn Syst 23(7):1148–1155CrossRef
45.
Feng R-B, Han Z-F, Wan WY, Leung C-S (2017) Properties and learning algorithms for faulty rbf networks with coexistence of weight and node failures. Neurocomputing 224:166–176CrossRef
46.
Konečnỳ J, Liu J, Richtárik P, Takáč M (2015) Mini-batch semi-stochastic gradient descent in the proximal setting. IEEE J Selected Top Signal Process 10(2):242–255CrossRef
47.
Qian Q, Jin R, Yi J, Zhang L, Zhu S (2015) Efficient distance metric learning by adaptive sampling and mini-batch stochastic gradient descent (sgd). Mach Learn 99(3):353–372MathSciNetMATHCrossRef
48.
Amari S-I (1993) Backpropagation and stochastic gradient descent method. Neurocomputing 5(4–5):185–196MATHCrossRef
49.
Li X, Orabona F (2019) On the convergence of stochastic gradient descent with adaptive stepsizes, In: The 22nd international conference on artificial intelligence and statistics, PMLR, pp 983–992
50.
Bottou L et al (1991) Stochastic gradient learning in neural networks. Proc Neuro-Nımes 91(8):12
51.
Cao Y, Gu Q (2019) Generalization bounds of stochastic gradient descent for wide and deep neural networks. Adv Neural Inform Process Syst 32:10836–10846
52.
Cha E, Leung C-S, Wong E (2020) Convergence of mini-batch learning for fault aware rbf networks. In: Yang H, Pasupa K, Leung AC-S, Kwok JT, Chan JH, King I (eds) Neural Information Processing. Springer International Publishing, Cham, pp 545–553CrossRef
53.
Chen S (2006) Local regularization assisted orthogonal least squares regression. Neurocomputing 69(4–6):559–585CrossRef
54.
Asuncion A, Newman D (2007) Uci machine learning repository
55.
Alcalá-Fdez J, Fernández A, Luengo J, Derrac J, García S, Sánchez L, Herrera F (2011) Keel data-mining software tool: data set repository, integration of algorithms and experimental analysis framework., Journal of Multiple-Valued Logic & Soft Computing 17
56.
Chang C-C, Lin C-J (2011) Libsvm: a library for support vector machines. ACM Trans Intell Syst Technol (TIST) 2(3):1–27CrossRef
Metadata
Title
Convergence analysis on the deterministic mini-batch learning algorithm for noise resilient radial basis function networks
Publication date
06-04-2022
Published in
International Journal of Machine Learning and Cybernetics / Issue 9/2022
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-022-01550-6

Other articles of this Issue 9/2022

International Journal of Machine Learning and Cybernetics 9/2022 Go to the issue

Original Article

Computing formal concepts in parallel via a workload rebalance approach

Original Article

Video person re-identification using key frame screening with index and feature reorganization based on inter-frame relation

Original Article

Brain tumor segmentation based on region of interest-aided localization and segmentation U-Net

Original Article

Multi-distance metric network for few-shot learning

Original Article

RD-NMSVM: neural mapping support vector machine based on parameter regularization and knowledge distillation

Original Article

Uncertain random portfolio optimization via semi-variance