Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
MTZ-Fachkongress

Antriebe und Energiesysteme von morgen


Austausch von Automobil- und Energiewirtschaft

Am 14./15. Mai geht es in Chemnitz um die Mobilitätswende durch Elektro- und Wasserstofffahrzeuge.
Erweiterte Suche
Anmelden
nach oben
Erschienen in:
Finding Near-Optimal Portfolios with Quality-Diversity Kapitel lesen Erstes Kapitel lesen

2024 | OriginalPaper | Buchkapitel

A Novel Two-Level Clustering-Based Differential Evolution Algorithm for Training Neural Networks

verfasst von : Seyed Jalaleddin Mousavirad, Diego Oliva, Gerald Schaefer, Mahshid Helali Moghadam, Mohammed El-Abd

Erschienen in: Applications of Evolutionary Computation

Verlag: Springer Nature Switzerland

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Determining appropriate weights and biases for feed-forward neural networks is a critical task. Despite the prevalence of gradient-based methods for training, these approaches suffer from sensitivity to initial values and susceptibility to local optima. To address these challenges, we introduce a novel two-level clustering-based differential evolution approach, C2L-DE, to identify the initial seed for a gradient-based algorithm. In the initial phase, clustering is employed to detect some regions in the search space. Population updates are then executed based on the information available within each region. A new central point is proposed in the subsequent phase, leveraging cluster centres for incorporation into the population. Our C2L-DE algorithm is compared against several recent DE-based neural network training algorithms, and is shown to yield favourable performance.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Vorheriges Kapitel Applying Graph Partitioning-Based Seeding Strategies to Software Modularisation
Nächstes Kapitel Iterated Beam Search for Wildland Fire Suppression
Fußnoten
Literatur
1.
2.
3.
4.
Bojnordi,E., Mousavirad,S.J., Schaefer, G., Korovin, I.: MCS-HMS: a multi-cluster selection strategy for the human mental search algorithm. In: IEEE Symposium Series on Computational Intelligence, pp. 1–6, 2021
5.
Cai, Z., Gong, W., Ling, C.X., Zhang, H.: A clustering-based differential evolution for global optimization. Appl. Soft Comput. 11(1), 1363–1379 (2011)CrossRef
6.
Deb, K.: A population-based algorithm-generator for real-parameter optimization. Soft. Comput. 9(4), 236–253 (2005)CrossRef
7.
Ding, S., Chunyang, S., Junzhao, Yu.: An optimizing BP neural network algorithm based on genetic algorithm. Artif. Intell. Rev. 36(2), 153–162 (2011)CrossRef
8.
Duan, H., Huang, L.: Imperialist competitive algorithm optimized artificial neural networks for UCAV global path planning. Neurocomputing 125, 166–171 (2014)CrossRef
9.
El-Bakry, M.Y., El-Dahshan, E.-S.A., El-Hamied, E.F.A.: Charged particle pseudorapidity distributions for Pb-Pb and Au-Au collisions using neural network model. Ukrainian J. Phys. 58(8), 709–709 (2013)CrossRef
10.
Fister, I., Fister, D., Deb, S., Mlakar, U., Brest, J.: Post hoc analysis of sport performance with differential evolution. Neural Comput. Appl. 32, 1–10 (2018)
11.
Hosaka, T.: Bankruptcy prediction using imaged financial ratios and convolutional neural networks. Expert Syst. Appl. 117, 287–299 (2019)CrossRef
12.
Ilonen, J., Kamarainen, J.-K., Lampinen, J.: Differential evolution training algorithm for feed-forward neural networks. Neural Process. Lett. 17(1), 93–105 (2003)CrossRef
13.
Kim, D., Kim, H., Chung, D.: A modified genetic algorithm for fast training neural networks. In: International Symposium on Neural Networks, pp. 660–665 (2005)
14.
Lera, G., Pinzolas, M.: Neighborhood based Levenberg-Marquardt algorithm for neural network training. IEEE Trans. Neural Networks 13(5), 1200–1203 (2002)CrossRef
15.
Levenberg, K.: A method for the solution of certain non-linear problems in least squares. Q. Appl. Math. 2(2), 164–168 (1944)MathSciNetCrossRef
16.
MacQueen, J.: Some methods for classification and analysis of multivariate observations. In: 5th Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
17.
Mandal, S., Saha, G., Pal, R.K.: Neural network training using firefly algorithm. Glob. J. Adv. Eng. Sci. 1(1), 7–11 (2015)
18.
Marquardt, D.W.: An algorithm for least-squares estimation of nonlinear parameters. J. Soc. Ind. Appl. Math. 11(2), 431–441 (1963)MathSciNetCrossRef
19.
20.
Moravvej, S.V., Mousavirad, S.J., Oliva, D., Schaefer, G., Sobhaninia, Z.: An improved DE algorithm to optimise the learning process of a BERT-based plagiarism detection model. In: IEEE Congress on Evolutionary Computation, pp. 1–7 (2022)
21.
Mousavirad, S.J., Bidgoli, A.A., Rahnamayan, S.: Tackling deceptive optimization problems using opposition-based DE with center-based Latin hypercube initialization. In: 14th International Conference on Computer Science and Education (2019)
22.
Mousavirad, S.J., Bidgoli, A.A., Ebrahimpour-Komleh, H., G.S.: A memetic imperialist competitive algorithm with chaotic maps for multi-layer neural network training. Int. J. Bio-Inspired Comput. 14(4), 227–236 (2019)
23.
Mousavirad, S.J., Bidgoli, A.A., Ebrahimpour-Komleh, H., Schaefer, G., Korovin, I.: An effective hybrid approach for optimising the learning process of multi-layer neural networks. In: International Symposium on Neural Networks, pp. 309–317 (2019)
24.
Mousavirad, S.J., Gandomi, A.H., Homayoun, H.: A clustering-based differential evolution boosted by a regularisation-based objective function and a local refinement for neural network training. In: IEEE Congress on Evolutionary Computation, pp. 1–8 (2022)
25.
Mousavirad, S.J., Jalali, S.M.J., Sajad, A., Abbas, K., Schaefer, G., Nahavandi, S.: Neural network training using a biogeography-based learning strategy. In: International Conference on Neural Information Processing (2020)
26.
Mousavirad, S.J., Oliva, D., Hinojosa, S., Schaefer, G.: Differential evolution-based neural network training incorporating a centroid-based strategy and dynamic opposition-based learning. In: IEEE Congress on Evolutionary Computation, pp. 1233–1240 (2021)
27.
Mousavirad, S.J., Rahmani, R., Dolatabadi, N.: A transfer learning based artificial neural network in geometrical design of textured surfaces for tribological applications. Surf. Topogr. Metrol. Prop. 11(2), 025001 (2023)CrossRef
28.
Mousavirad, S.J., Rahnamayan, S.: Evolving feedforward neural networks using a quasi-opposition-based differential evolution for data classification. In: IEEE Symposium Series on Computational Intelligence (2020)
29.
Mousavirad, S.J., Rahnamayan, S.: A novel center-based differential evolution algorithm. In: Congress on Evolutionary Computation (2020)
30.
Mousavirad, S.J., Schaefer, G., Jalali, S.M.J., Korovin, I.: A benchmark of recent population-based metaheuristic algorithms for multi-layer neural network training. In: Genetic and Evolutionary Computation Conference Companion, pp. 1402–1408 (2020)
31.
Mousavirad, S.J., Schaefer, G., Korovin, I.: An effective approach for neural network training based on comprehensive learning. In: International Conference on Pattern Recognition (2020)
32.
Mousavirad, S.J., Schaefer, G., Korovin, I., Oliva, D.: RDE-OP: a region-based differential evolution algorithm incorporation opposition-based learning for optimising the learning process of multi-layer neural networks. In: Castillo, P.A., Jiménez Laredo, J.L. (eds.) EvoApplications 2021. LNCS, vol. 12694, pp. 407–420. Springer, Cham (2021). https://​doi.​org/​10.​1007/​978-3-030-72699-7_​26CrossRef
33.
Munkhdalai, L., Lee, J.Y., Ryu, K.H.: A hybrid credit scoring model using neural networks and logistic regression. In: Pan, J.-S., Li, J., Tsai, P.-W., Jain, L.C. (eds.) Advances in Intelligent Information Hiding and Multimedia Signal Processing. SIST, vol. 156, pp. 251–258. Springer, Singapore (2020). https://​doi.​org/​10.​1007/​978-981-13-9714-1_​27CrossRef
34.
Nawi, N.M., khan, A., Rehman, M.Z., Aziz, M.A., Herawan, T., Abawajy, J.H.: An accelerated particle swarm optimization based Levenberg Marquardt back propagation algorithm. In: Loo, C.K., Yap, K.S., Wong, K.W., Teoh, A., Huang, K. (eds.) ICONIP 2014. LNCS, vol. 8835, pp. 245–253. Springer, Cham (2014). https://​doi.​org/​10.​1007/​978-3-319-12640-1_​30CrossRef
35.
Pedram, M., Mousavirad, S.J., Schaefer, G.: Training neural networks with Lévy flight distribution algorithm. In: 7th International Conference on Harmony Search, Soft Computing and Applications, pp. 93–103 (2022)
36.
Rahmani, S., Mousavirad, S.J., El-Abd, M., Schaefer, G., Oliva, D.: Centroid-based differential evolution with composite trial vector generation strategies for neural network training. In: Correia, J., Smith, S., Qaddoura, R. (eds.) International Conference on the Applications of Evolutionary Computation, vol. 13989, pp. 608–622. Springer, Cham (2023). https://​doi.​org/​10.​1007/​978-3-031-30229-9_​39
37.
Sexton, R.S., Gupta, J.N.D.: Comparative evaluation of genetic algorithm and backpropagation for training neural networks. Inform. Sci. 129(1–4), 45–59 (2000)CrossRef
38.
Shi, Y., Eberhart, R.: A modified particle swarm optimizer. In: IEEE International Conference on Evolutionary Computation, pp. 69–73 (1998)
39.
Si, T., Dutta, R.: Partial opposition-based particle swarm optimizer in artificial neural network training for medical data classification. Int. J. Inform. Technol. Decis. Making 18(5), 1717–1750 (2019)CrossRef
40.
Storn, R., Price, K.: Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J. Global Optim. 11(4), 341–359 (1997)MathSciNetCrossRef
41.
Wang, X., et al.: Massive expansion and differential evolution of small heat shock proteins with wheat (triticum aestivum l.) polyploidization. Sci. Rep. 7(1), 1–12 (2017)
Metadaten
Titel
A Novel Two-Level Clustering-Based Differential Evolution Algorithm for Training Neural Networks
verfasst von
Seyed Jalaleddin Mousavirad
Diego Oliva
Gerald Schaefer
Mahshid Helali Moghadam
Mohammed El-Abd
Copyright-Jahr
2024
Buch
Applications of Evolutionary Computation

Print ISBN: 978-3-031-56851-0

Electronic ISBN: 978-3-031-56852-7

Copyright-Jahr: 2024

DOI
https://doi.org/10.1007/978-3-031-56852-7_17

Premium Partner

    Bildnachweise