nach oben

Erschienen in:

2021 | OriginalPaper | Buchkapitel

13. Training Multi-layer Perceptron Using Hybridization of Chaotic Gravitational Search Algorithm and Particle Swarm Optimization

verfasst von : Sajad Ahmad Rather, P. Shanthi Bala, Pillai Lekshmi Ashokan

Erschienen in: Applying Particle Swarm Optimization

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

A novel amalgamation strategy, namely, chaotic gravitational search algorithm (CGSA) and particle swarm optimization (PSO), has been employed for training multi-layer perceptron (MLP) neural network. It is called CGSAPSO. In CGSAPSO, exploration is carried out by CGSA, and exploitation is performed using PSO. The sigmoid activation function is utilized for training MLP. Besides, a matrix encoding strategy has been used for providing a synergy between neural biases, weights, and CGSAPSO searcher agents. To validate the effectiveness of the hybrid framework, CGSAPSO is applied to three different classification datasets, namely, XOR, Iris, and Balloon. The investigation of results is carried out through various performance metrics like average, standard deviation, median, convergence speed, execution time, and classification rate analysis. Besides, a pair-wise non-parametric signed Wilcoxon rank-sum test has also been conducted for statistical verification of simulation results. In addition, the numerical outcomes of CGSAPSO are also compared with standard GSA, PSO, and hybrid PSOGSA. The experimental results indicate that CGSAPSO provides better results in the form of recognition accuracy and global optima as compared to competing algorithms.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Particle Swarm Optimization in Global Path Planning for Swarm of Robots

Nächstes Kapitel Solving Optimization Problem with Particle Swarm Optimization: Solving Hybrid Flow Shop Scheduling Problem with Particle Swarm Optimization Algorithm

Altay, E. V., & Alatas, B. (2020). Bird swarm algorithms with chaotic mapping. Artificial Intelligence Review, 53(2), 1373–1414. https://doi.org/10.1007/s10462-019-09704-9.CrossRef

Bebis, G., & Georgiopoulos, M. (1994). Feed-forward neural networks. IEEE Potentials, 13(4), 27–31.CrossRef

Blake, C., & Merz, C. J. (1998). UCI: Repository of machine learning databases. Retrieved from http://www.ics.uci.edu/~mlearn/MLRepository.html.

Derrac, J., García, S., Molina, D., & Herrera, F. (2011). A practical tutorial on the use of nonparametric statistical tests as a methodology for comparing evolutionary and swarm intelligence algorithms. Swarm and Evolutionary Computation, 1(1), 3–18.CrossRef

Dorffner, G. (1996). Neural networks for time series processing. Neural Network World, 6, 447–468.

Dorigo, M., Birattari, M., & Stutzle, T. (2006). Ant colony optimization. IEEE Computational Intelligence Magazine, 1(4), 28–39. https://doi.org/10.1109/MCI.2006.329691.CrossRef

Gandomi, A. H., & Yang, X. S. (2014). Chaotic bat algorithm. Journal of Computational Science, 5(2), 224–232. https://doi.org/10.1016/j.jocs.2013.10.002.CrossRef

Green, R. C., II, Wang, L., & Alam, M. (2012). Training neural networks using central force optimization and particle swarm optimization: Insights and comparisons. Expert Systems with Applications, 39(1), 555–563.CrossRef

Gudise, V. G., & Venayagamoorthy, G. K. (2003). Comparison of particle swarm optimization and backpropagation as training algorithms for neural networks. In Proceedings of the 2003 IEEE Swarm Intelligence Symposium, pp. 110–117.

Halliday, D., Resnick, R., & Walker, J. (2000). Fundamentals of physics (6th Edition). Delhi: Wiley.

Huang, M. L., & Chou, Y. C. (2019). Combining a gravitational search algorithm, particle swarm optimization, and fuzzy rules to improve the classification performance of a feed-forward neural network. Computer Methods and Programs in Biomedicine, 180, 105–116. https://doi.org/10.1016/j.cmpb.2019.CrossRef

Hussain, K., Salleh, M. N. M., Cheng, S., & Shi, Y. (2019). On the exploration and exploitation in popular swarm-based metaheuristic algorithms. Neural Computing and Applications, 31(11), 7665–7683. https://doi.org/10.1007/s00521-018-3592-0.CrossRef

Itano, F., DeSousa, M. A. D. A., & Hernandez, E. D. M. (2018). Extending MLP ANN hyperparameters optimization by using genetic algorithm. In Proceedings of the IEEE 2018 International Joint Conference on Neural Networks (IJCNN), pp. 1-8.

Jacobs, R. A. (1988). Increased rates of convergence through learning rate adaptation. Neural Networks, 1(4), 295–307.CrossRef

James, J. Q., Lam, A. Y., & Li, V. O. (2011, June). Evolutionary artificial neural network based on chemical reaction optimization. In 2011 IEEE congress of evolutionary computation (CEC) (pp. 2083–2090). New York: IEEE.

Karaboga, D., Akay, B., & Ozturk, C. (2007). Artificial bee colony (ABC) optimization algorithm for training feed-forward neural networks. In Springer international conference on modeling decisions for artificial intelligence (pp. 318–329). Berlin: Springer.CrossRef

Kennedy, J., & Eberhart, R. (1995). Particle swarm optimization. In Proceedings of ICNN ’95-IEEE International conference on neural networks (pp. 1942–1948). New York: IEEE.CrossRef

Khishe, M., & Mosavi, M. R. (2020). Classification of underwater acoustical dataset using neural network trained by chimp optimization algorithm. Applied Acoustics, 157, 107005. https://doi.org/10.1016/j.apacoust.2019.107005.CrossRef

Llonen, J., Kamarainen, J. K., & Lampinen, J. (2003). Differential evolution training algorithm for feed-forward neural networks. Neural Processing Letters, 17(1), 93–105.CrossRef

Markowitz, H. (1952). Portfolio selection. Journal of Finance, 79(1), 77–91.

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. Bulletin of Mathematical Biophysics, 5(4), 115–133.CrossRef

Mejía-de-Dios, J. A., & Mezura-Montes, E. (2019). A new evolutionary optimization method based on center of mass. Decision Science in Action, 2019, 65–74. https://doi.org/10.1007/978-981-13-0860-4_6.CrossRef

Mendes, R., Cortez, P., Rocha, M., & Neves, J. (2002). Particle swarms for feed-forward neural network training. Proceedings of the IEEE International Joint Conference on Neural Networks, 6, 1895–1899.

Mirjalili, S. (2015). How effective is the Grey Wolf optimizer in training multi-layer perceptrons. Applied Intelligence, 43(1), 150–161. https://doi.org/10.1007/s10489-014-0645-7.CrossRef

Mirjalili, S., & Gandomi, A. H. (2017). Chaotic gravitational constants for the gravitational search algorithm. Applied Soft Computing, 53, 407–419. https://doi.org/10.1016/j.asoc.2017.01.008.CrossRef

Mirjalili, S., & Hashim, S. Z. M. (2010). A new hybrid PSOGSA algorithm for function optimization. In 2010 IEEE International Conference on Computer and Information Application, 2010, 374–377.CrossRef

Mirjalili, S., & Lewis, A. (2016). The whale optimization algorithm. Advances in Engineering Software, 95, 51–67. https://doi.org/10.1016/j.advengsoft.2016.01.008.CrossRef

Mirjalili, S., Mirjalili, S. M., & Lewis, A. (2014). Let a biogeography-based optimizer train your multi-layer perceptron. Information Sciences, 269, 188–209. https://doi.org/10.1016/j.ins.2014.01.038.CrossRef

Mirjalili, S., Mirjalili, S. M., & Yang, X. S. (2014). Binary bat algorithm. Neural Computing and Applications, 25(3-4), 663–681. https://doi.org/10.1007/s00521-013-1525-5.CrossRef

Mirjalili, S., Mohd Hashim, S. Z., & Moradian Sardroudi, H. (2012). Training Feed-forward neural networks using hybrid particle swarm optimization and gravitational search algorithm. Applied Mathematics and Computation, 218(22), 11125–11137.CrossRef

Mirjalili, S. M., Abedi, K., & Mirjalili, S. (2013). Optical buffer performance enhancement using particle swarm optimization in ring-shape-hole photonic crystal waveguide. The Optician, 124(23), 5989–5993.

Nawi, N. M., Ransing, R. S., Salleh, M. N. M., Ghazali, R., & Hamid, N. A. (2010). An improved back propagation neural network algorithm on classification problems. In Database theory and application, bio-science and bio-technology (pp. 177–188). https://doi.org/10.1007/978-3-642-17622-7_18.CrossRef

Ooyen, A., & Nienhuis, B. (1992). Improving the convergence of the backpropagation algorithm. Neural Networks, 5(3), 465–471.CrossRef

Pereira, L. A., Afonso, L. C., Papa, J. P., Vale, Z. A., Ramos, C. C., Gastaldello, D. S., & Souza, A. N. (2013, April). Multilayer perceptron neural networks training through charged system search and its application for non-technical losses detection. In 2013 IEEE PES Conference on Innovative Smart Grid Technologies (ISGT Latin America) (pp. 1–6). New York: IEEE.

Pereira, L. A., Rodrigues, D., Ribeiro, P. B., Papa, J. P., & Weber, S. A. (2014, May). Social-spider optimization-based artificial neural networks training and its applications for Parkinson’s disease identification. In 2014 IEEE 27th international symposium on computer-based medical systems (pp. 14–17). New York: IEEE.CrossRef

Rashedi, E., Nezamabadi-pour, H., & Saryazdi, S. (2009). GSA: A gravitational search algorithm. Information Sciences, 179(13), 2232–2248.CrossRef

Rather, S. A., & Bala, P. S. (2019a). A holistic review on gravitational search algorithm and its hybridization with other algorithms. In 2019 IEEE International conference on electrical, computer and communication technologies (ICECCT) (pp. 1–6). New York: IEEE. https://doi.org/10.1109/ICECCT.2019.8869279.CrossRef

Rather, S. A., & Bala, P. S. (2019b). Analysis of gravitation based optimization algorithms for clustering and classification. In Handbook of research on big data clustering and machine learning (pp. 77–99). Hershey: IGI Global. https://doi.org/10.4018/978-1-7998-0106-1.ch005.CrossRef

Rather, S. A., & Bala, P. S. (2019c). Hybridization of constriction coefficient based particle swarm optimization and gravitational search algorithm for function optimization. In 2019 Elsevier International Conference on Advances in Electronics, Electrical, and Computational Intelligence (ICAEEC-2019). Amsterdam: Elsevier. https://doi.org/10.2139/ssrn.3576489.CrossRef

Rather, S. A., & Bala, P. S. (2020a). A hybrid constriction coefficient based particle swarm optimization and gravitational search algorithm for training multi-layer perceptron (MLP). International Journal of Intelligent Computing and Cybernetics, 13(2), 129–165. https://doi.org/10.1108/JICC-09-2019-0105.CrossRef

Rather, S. A., & Bala, P. S. (2020b). Swarm-based chaotic gravitational search algorithm for solving mechanical engineering design problems. World Journal of Engineering, 17(1), 97–114. https://doi.org/10.1108/WJE-09-2019-0254.CrossRef

Rather, S. A., & Sharma, N. (2017). GSA-BBO hybridization algorithm. International Journal of Advance Research in Science and Engineering, 6, 596–608.

Saremi, S., Mirjalili, S., & Lewis, A. (2014). Biogeography-based optimization with chaos. Neural Computing and Applications, 25(5), 1077–1097.CrossRef

Simon, D. (2008). Biogeography‐based optimization. IEEE Transactions on Evolutionary Computation, 12, 702–713.CrossRef

Weir, M. K. (1991). A method doe self-determination of adaptive learning rates in backpropagation. Neural Networks, 4(3), 371–379.CrossRef

Wilcoxon, F. (1945). Individual comparisons by ranking methods. Biometrics Bulletin, 1(6), 80–83.CrossRef

Yang, X., & Gandomi, A. H. (2012). Bat algorithm: A novel approach for global engineering optimization. Engineering Computations, 29(5), 464–483. https://doi.org/10.1108/02644401211235834.CrossRef

Clerc, M., & Kennedy, J. (2002). The particle swarm-explosion, stability, and convergence in a multidimensional complex space. IEEE Transactions on Evolutionary Computation, 6(1), 58–73.

Kalinlia, A., & Karabogab, N. (2005). Artificial immune algorithm for IIR filter design Engineering. Applications of Artificial Intelligence, 18, 919–929.CrossRef

Kirkpatrick, S., Gelatto, C. D., & Vecchi, M. P. (1983). Optimization by simulated annealing. Science, 220, 671–680.CrossRef

Lavika, S., Sharthak, & Satyajit. (2016). Hybridization of gravitational search algorithm and biogeography based optimization and its application on grid scheduling problem. Ninth International Conference on Contemporary Computing (IC3), 2016, 1–6.

Li, C., & Zhou, J. (2011). Parameters identification of hydraulic turbine governing system using improved gravitational search algorithm. Energy Conversion and Management, 52, 374–381.CrossRef

Svozil, D., Kvasnicka, V., & Pospichal, J. (1997). Introduction to multi-layer feed-forward neural networks. Chemometrics and Intelligent Laboratory Systems, 39(1), 43–62. https://doi.org/10.1016/S0169-7439(97)00061-0.

Tang, K. S., Man, K. F., Kwong, S., & He, Q. (1996). Genetic algorithms and their applications. IEEE Signal Processing Magazine, 13, 22–37.CrossRef

Wolpert, D. H., & Macready, W. G. (1997). No free lunch theorems for optimization. IEEE Transactions on Evolutionary Computation, 1(1), 67–82.CrossRef

Yang, L. J., & Chen, T. L. (2002). Application of chaos in genetic algorithms. Communications on Theoretical Physics, 38(2), 168–172.CrossRef

Walczak, S., & Cerpa, N. (1999). Heuristic principles for the design of artificial neural networks. Information and Software Technology, 41(2), 107–117. https://doi.org/10.1016/S0950-5849(98)00116-5.

Titel: Training Multi-layer Perceptron Using Hybridization of Chaotic Gravitational Search Algorithm and Particle Swarm Optimization
verfasst von: Sajad Ahmad Rather
P. Shanthi Bala
Pillai Lekshmi Ashokan
Verlag: Springer International Publishing
Buch: Applying Particle Swarm Optimization
Print ISBN: 978-3-030-70280-9

Electronic ISBN: 978-3-030-70281-6

Copyright-Jahr: 2021
DOI: https://doi.org/10.1007/978-3-030-70281-6_13