Skip to main content
Erschienen in: New Generation Computing 1/2022

24.03.2022

Hyperparameter Optimization of Evolving Spiking Neural Network for Time-Series Classification

verfasst von: Tasbiha Ibad, Said Jadid Abdulkadir, Norshakirah Aziz, Mohammed Gamal Ragab, Qasem Al-Tashi

Erschienen in: New Generation Computing | Ausgabe 1/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Spiking neural networks are the third generation of artificial neural networks that are inspired by a new brain-inspired computational model of ANN. Spiking neural network encodes and processes neural information through precisely timed spike trains. eSNN is an enhanced version of SNN, motivated by the principles of Evolving Connectionist System (ECoS), which is relatively a new classifier in the neural information processing area. The performance of eSNN is highly influenced by the values of its significant hyperparameters’ modulation factor (mod), threshold factor (c), and similarity factor (sim). In contrast to the manual tuning of hyperparameters, automated tuning is more reliable. Therefore, this research presents an optimizer-based eSNN architecture, intended to solve the issue regarding optimum hyperparameters’ values’ selection of eSNN. The proposed model is named eSNN-SSA where SSA stands for salp swarm algorithm, which is a metaheuristic optimization technique integrated with eSNN architecture. For the integration of eSNN-SSA, Thorpe’s standard model of eSNN is used with population rate encoding. To examine the performance of eSNN-SSA, various benchmarking data sets from the UCR/UAE time-series classification repository are utilized. From the experimental results, it is concluded that the salp swarm algorithm plays an effective role in improving the flexibility of the eSNN. The proposed eSNN-SSA offers solutions to conquer the disadvantages of eSNN in determining the best number of pre-synaptic neurons for time-series classification problems. The performance accuracy obtained by eSNN-SSA was on datasets spoken Arabic digits, articulatory word recognition, character trajectories, wafer, and GunPoint, i.e., 0.96, 0.97, 0.94, 1.0, and 0.94, respectively. The proposed approach outperformed standard eSNN in terms of time complexity.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Basu, J.K., Bhattacharyya, D., Kim, T.: Use of artificial neural network in pattern recognition. Int. J. Soft. Eng. Appl. 4(2) (2010) Basu, J.K., Bhattacharyya, D., Kim, T.: Use of artificial neural network in pattern recognition. Int. J. Soft. Eng. Appl. 4(2) (2010)
2.
Zurück zum Zitat Zainuddin, Z., Ong, P.: Function approximation using artificial neural networks. WSEAS Trans. Math. 7(6), 333–338 (2008)MathSciNet Zainuddin, Z., Ong, P.: Function approximation using artificial neural networks. WSEAS Trans. Math. 7(6), 333–338 (2008)MathSciNet
3.
Zurück zum Zitat Beskopylny, A., Lyapin, A., Anysz, H., Meskhi, B., Veremeenko, A., Mozgovoy, A.: Artificial neural networks in classification of steel grades based on non-destructive tests. Materials (Basel) 13(11), 2445 (2020)CrossRef Beskopylny, A., Lyapin, A., Anysz, H., Meskhi, B., Veremeenko, A., Mozgovoy, A.: Artificial neural networks in classification of steel grades based on non-destructive tests. Materials (Basel) 13(11), 2445 (2020)CrossRef
4.
Zurück zum Zitat Heo, S., Lee, J.H.: Fault detection and classification using artificial neural networks. IFAC-PapersOnLine 51(18), 470–475 (2018)CrossRef Heo, S., Lee, J.H.: Fault detection and classification using artificial neural networks. IFAC-PapersOnLine 51(18), 470–475 (2018)CrossRef
5.
Zurück zum Zitat Hagan, M.T., Demuth, H.B., Beale, M.: Neural Network Design. PWS Publishing Co., Boston (1997) Hagan, M.T., Demuth, H.B., Beale, M.: Neural Network Design. PWS Publishing Co., Boston (1997)
6.
Zurück zum Zitat Vreeken, J.: Spiking Neural Networks, an Introduction. Utrecht University Information and Computing Sciences, Utrecht (2003) Vreeken, J.: Spiking Neural Networks, an Introduction. Utrecht University Information and Computing Sciences, Utrecht (2003)
7.
Zurück zum Zitat Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)CrossRef Maass, W.: Networks of spiking neurons: the third generation of neural network models. Neural Netw. 10(9), 1659–1671 (1997)CrossRef
9.
Zurück zum Zitat Ahmed, F.Y., Yusob, B., Hamed, H.N.A.: Computing with spiking neuron networks: a review. Int. J. Adv. Soft. Comput. Appl. 6(1) (2014) Ahmed, F.Y., Yusob, B., Hamed, H.N.A.: Computing with spiking neuron networks: a review. Int. J. Adv. Soft. Comput. Appl. 6(1) (2014)
10.
Zurück zum Zitat Agebure, M.A., Wumnaya, P.A., Baagyere, E.Y.: A survey of supervised learning models for spiking neural network. Networks 5 (2021) Agebure, M.A., Wumnaya, P.A., Baagyere, E.Y.: A survey of supervised learning models for spiking neural network. Networks 5 (2021)
11.
Zurück zum Zitat Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge (1992)CrossRef Holland, J.H.: Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence. MIT Press, Cambridge (1992)CrossRef
12.
Zurück zum Zitat Kasabov, N.K.: The ECOS framework and the ECO learning method for evolving connectionist systems. J. Adv. Comput. Intell. Intell. Inform. 2(6), 195–202 (1998)CrossRef Kasabov, N.K.: The ECOS framework and the ECO learning method for evolving connectionist systems. J. Adv. Comput. Intell. Intell. Inform. 2(6), 195–202 (1998)CrossRef
13.
Zurück zum Zitat Saleh, A.Y., Hameed, H., Najib, M., Salleh, M.: A novel hybrid algorithm of differential evolution with evolving spiking neural network for pre-synaptic neurons optimization. Int. J. Adv. Soft Comput. Appl 6(1), 1–16 (2014) Saleh, A.Y., Hameed, H., Najib, M., Salleh, M.: A novel hybrid algorithm of differential evolution with evolving spiking neural network for pre-synaptic neurons optimization. Int. J. Adv. Soft Comput. Appl 6(1), 1–16 (2014)
14.
Zurück zum Zitat Abdull Hamed, H.N.: Novel integrated methods of evolving spiking neural network and particle swarm optimisation. Auckland University of Technology, Auckland (2012) Abdull Hamed, H.N.: Novel integrated methods of evolving spiking neural network and particle swarm optimisation. Auckland University of Technology, Auckland (2012)
15.
Zurück zum Zitat Kennedy, J.: Swarm intelligence. In: Handbook of nature-inspired and innovative computing, pp. 187–219. Springer, Berlin (2006)CrossRef Kennedy, J.: Swarm intelligence. In: Handbook of nature-inspired and innovative computing, pp. 187–219. Springer, Berlin (2006)CrossRef
16.
Zurück zum Zitat Anderson, P.A.V., Bone, Q.: Communication between individuals in salp chains. II. Physiology. Proc. R. Soc. London. Ser. B. Biol. Sci. 210(1181), 559–574 (1980) Anderson, P.A.V., Bone, Q.: Communication between individuals in salp chains. II. Physiology. Proc. R. Soc. London. Ser. B. Biol. Sci. 210(1181), 559–574 (1980)
17.
Zurück zum Zitat Ruiz, A.P., Flynn, M., Large, J., Middlehurst, M., Bagnall, A.: The great multivariate time-series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 35(2), 401–449 (2021)MathSciNetCrossRef Ruiz, A.P., Flynn, M., Large, J., Middlehurst, M., Bagnall, A.: The great multivariate time-series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 35(2), 401–449 (2021)MathSciNetCrossRef
18.
Zurück zum Zitat Roslan, F., Hamed, H.N.A., Isa, M.A.: The enhancement of evolving spiking neural network with firefly algorithm. J. Telecommun. Electron. Comput. Eng. 9(3–3), 63–66 (2017) Roslan, F., Hamed, H.N.A., Isa, M.A.: The enhancement of evolving spiking neural network with firefly algorithm. J. Telecommun. Electron. Comput. Eng. 9(3–3), 63–66 (2017)
19.
Zurück zum Zitat Yusuf, Z.M., Hamed, H.N.A., Yusuf, L.M., Isa, M.A.: Evolving spiking neural network (ESNN) and harmony search algorithm (HSA) for parameter optimization. In: 2017 6th International Conference on Electrical Engineering and Informatics (ICEEI), pp. 1–6 (2017) Yusuf, Z.M., Hamed, H.N.A., Yusuf, L.M., Isa, M.A.: Evolving spiking neural network (ESNN) and harmony search algorithm (HSA) for parameter optimization. In: 2017 6th International Conference on Electrical Engineering and Informatics (ICEEI), pp. 1–6 (2017)
20.
Zurück zum Zitat Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A., Siong, T.C., Othman, M.K.: A new harmony search algorithm with evolving spiking neural network for classification problems. J. Telecommun. Electron. Comput. Eng. 9(3–11), 23–26 (2017) Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A., Siong, T.C., Othman, M.K.: A new harmony search algorithm with evolving spiking neural network for classification problems. J. Telecommun. Electron. Comput. Eng. 9(3–11), 23–26 (2017)
21.
Zurück zum Zitat Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A.: Multi-objective differential evolution of evolving spiking neural networks for classification problems. In: IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 351–368 (2015) Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A.: Multi-objective differential evolution of evolving spiking neural networks for classification problems. In: IFIP International Conference on Artificial Intelligence Applications and Innovations, pp. 351–368 (2015)
22.
Zurück zum Zitat John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: Machine Learning Proceedings 1994. Elsevier, pp. 121–129 (1994) John, G.H., Kohavi, R., Pfleger, K.: Irrelevant features and the subset selection problem. In: Machine Learning Proceedings 1994. Elsevier, pp. 121–129 (1994)
23.
Zurück zum Zitat Hamed, H.N.A., Saleh, A.Y., Shamsuddin, S.M., Ibrahim, A.O.: Multi-objective K-means evolving spiking neural network model based on differential evolution. In: 2015 International Conference on Computing, Control, Networking, Electronics and Embedded Systems Engineering (ICCNEEE), pp. 379–383 (2015) Hamed, H.N.A., Saleh, A.Y., Shamsuddin, S.M., Ibrahim, A.O.: Multi-objective K-means evolving spiking neural network model based on differential evolution. In: 2015 International Conference on Computing, Control, Networking, Electronics and Embedded Systems Engineering (ICCNEEE), pp. 379–383 (2015)
24.
Zurück zum Zitat Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A.: Memetic harmony search algorithm based on multi-objective differential evolution of evolving spiking neural networks. Int. J. Swarm Intel. Evol. Comput. 5(130), 2 (2016) Saleh, A.Y., Shamsuddin, S.M., Hamed, H.N.A.: Memetic harmony search algorithm based on multi-objective differential evolution of evolving spiking neural networks. Int. J. Swarm Intel. Evol. Comput. 5(130), 2 (2016)
25.
Zurück zum Zitat Saleh, A.Y., Hamed, H.N.B.A., Shamsuddin, S.M., Ibrahim, A.O.: A new hybrid k-means evolving spiking neural network model based on differential evolution. In: International Conference of Reliable Information and Communication Technology, pp. 571–583 (2017) Saleh, A.Y., Hamed, H.N.B.A., Shamsuddin, S.M., Ibrahim, A.O.: A new hybrid k-means evolving spiking neural network model based on differential evolution. In: International Conference of Reliable Information and Communication Technology, pp. 571–583 (2017)
26.
Zurück zum Zitat Bohte, S.M., Kok, J.N., La Poutre, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4), 17–37 (2002)CrossRef Bohte, S.M., Kok, J.N., La Poutre, H.: Error-backpropagation in temporally encoded networks of spiking neurons. Neurocomputing 48(1–4), 17–37 (2002)CrossRef
27.
Zurück zum Zitat Séguier, R., Mercier, D.: Audio-visual speech recognition one pass learning with spiking neurons. In: International Conference on Artificial Neural Networks, pp. 1207–1212 (2002) Séguier, R., Mercier, D.: Audio-visual speech recognition one pass learning with spiking neurons. In: International Conference on Artificial Neural Networks, pp. 1207–1212 (2002)
28.
Zurück zum Zitat Kasabov, N.K.: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence. Springer, Berlin (2019)CrossRef Kasabov, N.K.: Time-Space, Spiking Neural Networks and Brain-Inspired Artificial Intelligence. Springer, Berlin (2019)CrossRef
29.
Zurück zum Zitat Kasabov, N.: Integrative connectionist learning systems inspired by nature: current models, future trends and challenges. Nat. Comput. 8(2), 199–218 (2009)MathSciNetCrossRef Kasabov, N.: Integrative connectionist learning systems inspired by nature: current models, future trends and challenges. Nat. Comput. 8(2), 199–218 (2009)MathSciNetCrossRef
30.
Zurück zum Zitat Schliebs, S., Kasabov, N.: Computational modeling with spiking neural networks, pp. 625–646. Springer Handb. Bio-/neuroinformatics, Berlin (2014) Schliebs, S., Kasabov, N.: Computational modeling with spiking neural networks, pp. 625–646. Springer Handb. Bio-/neuroinformatics, Berlin (2014)
31.
Zurück zum Zitat Mirjalili, S., Gandomi, A.H., Mirjalili, S.Z., Saremi, S., Faris, H., Mirjalili, S.M.: Salp swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv. Eng. Softw. 114, 163–191 (2017)CrossRef Mirjalili, S., Gandomi, A.H., Mirjalili, S.Z., Saremi, S., Faris, H., Mirjalili, S.M.: Salp swarm algorithm: a bio-inspired optimizer for engineering design problems. Adv. Eng. Softw. 114, 163–191 (2017)CrossRef
32.
Zurück zum Zitat Abualigah, L., Shehab, M., Alshinwan, M., Alabool, H.: Salp swarm algorithm: a comprehensive survey. Neural Comput. Appl. 32(15), 11195–11215 (2020)CrossRef Abualigah, L., Shehab, M., Alshinwan, M., Alabool, H.: Salp swarm algorithm: a comprehensive survey. Neural Comput. Appl. 32(15), 11195–11215 (2020)CrossRef
33.
Zurück zum Zitat Ibrahim, R.A., Ewees, A.A., Oliva, D., Abd Elaziz, M., Lu, S.: Improved salp swarm algorithm based on particle swarm optimization for feature selection. J. Ambient Intell. Humaniz. Comput. 10(8), 3155–3169 (2019)CrossRef Ibrahim, R.A., Ewees, A.A., Oliva, D., Abd Elaziz, M., Lu, S.: Improved salp swarm algorithm based on particle swarm optimization for feature selection. J. Ambient Intell. Humaniz. Comput. 10(8), 3155–3169 (2019)CrossRef
34.
Zurück zum Zitat Hamed, H.N.A., Kasabov, N., Shamsuddin, S.M.: Integrated feature selection and parameter optimization for evolving spiking neural networks using quantum inspired particle swarm optimization. In: 2009 International Conference of Soft Computing and Pattern Recognition, pp. 695–698 (2009) Hamed, H.N.A., Kasabov, N., Shamsuddin, S.M.: Integrated feature selection and parameter optimization for evolving spiking neural networks using quantum inspired particle swarm optimization. In: 2009 International Conference of Soft Computing and Pattern Recognition, pp. 695–698 (2009)
35.
Zurück zum Zitat Dau, H.A., et al.: The UCR time-series archive. IEEE/CAA J. Autom. Sin. 6(6), 1293–1305 (2019)CrossRef Dau, H.A., et al.: The UCR time-series archive. IEEE/CAA J. Autom. Sin. 6(6), 1293–1305 (2019)CrossRef
36.
Zurück zum Zitat Forrester, A., Sobester, A., Keane, A.: Engineering design via surrogate modelling: a practical guide. Wiley, New York (2008)CrossRef Forrester, A., Sobester, A., Keane, A.: Engineering design via surrogate modelling: a practical guide. Wiley, New York (2008)CrossRef
37.
Zurück zum Zitat Al-Tashi, Q., Rais, H., Jadid Abdulkadir, S., Mirjalili, S.: Feature selection based on grey wolf optimizer for oil and gas reservoir classification. In: 2020 International Conference on Computational Intelligence (ICCI). IEEE, pp. 211–216 (2020) Al-Tashi, Q., Rais, H., Jadid Abdulkadir, S., Mirjalili, S.: Feature selection based on grey wolf optimizer for oil and gas reservoir classification. In: 2020 International Conference on Computational Intelligence (ICCI). IEEE, pp. 211–216 (2020)
38.
Zurück zum Zitat Ren, H., Li, J., Chen, H., Li, C.: Adaptive levy-assisted salp swarm algorithm: analysis and optimization case studies. Math. Comput. Simul. 181, 380–409 (2021)MathSciNetCrossRef Ren, H., Li, J., Chen, H., Li, C.: Adaptive levy-assisted salp swarm algorithm: analysis and optimization case studies. Math. Comput. Simul. 181, 380–409 (2021)MathSciNetCrossRef
39.
Zurück zum Zitat Ragab, M.G., Abdulkadir, S.J., Aziz, N., Al-Tashi, Q., Alyousifi, Y., Alhussian, H., Alqushaibi, A.: A novel one-dimensional CNN with exponential adaptive gradients for air pollution index prediction. Sustainability. 12(23), 10090 (2020). https://doi.org/10.3390/su122310090CrossRef Ragab, M.G., Abdulkadir, S.J., Aziz, N., Al-Tashi, Q., Alyousifi, Y., Alhussian, H., Alqushaibi, A.: A novel one-dimensional CNN with exponential adaptive gradients for air pollution index prediction. Sustainability. 12(23), 10090 (2020). https://​doi.​org/​10.​3390/​su122310090CrossRef
40.
Zurück zum Zitat Abdulkadir, S.J., Shamsuddin, S.M., Sallehuddin, R.: Three term back propagation network for moisture prediction. In: International Conference on Clean and Green Energy, pp. 103–107 (2012) Abdulkadir, S.J., Shamsuddin, S.M., Sallehuddin, R.: Three term back propagation network for moisture prediction. In: International Conference on Clean and Green Energy, pp. 103–107 (2012)
41.
Zurück zum Zitat Abdulkadir, S.J., Alhussian, H., Alzahrani, A.I.: Analysis of recurrent neural networks for henon simulated time-series forecasting. J. Telecommun. Electron. Comput. Eng. 10(1–8), 155–159 (2018) Abdulkadir, S.J., Alhussian, H., Alzahrani, A.I.: Analysis of recurrent neural networks for henon simulated time-series forecasting. J. Telecommun. Electron. Comput. Eng. 10(1–8), 155–159 (2018)
42.
Zurück zum Zitat Alhussian, H., Zakaria, N., Patel, A., Jaradat, A., Abdulkadir, S.J., Ahmed, A.Y., Bahbouh, H.T., Fageeri, S.O., Elsheikh, A.A., Watada, J.: Investigating the schedulability of periodic real-time tasks in virtualized cloud environment. IEEE Access 7, 29533–29542 (2019)CrossRef Alhussian, H., Zakaria, N., Patel, A., Jaradat, A., Abdulkadir, S.J., Ahmed, A.Y., Bahbouh, H.T., Fageeri, S.O., Elsheikh, A.A., Watada, J.: Investigating the schedulability of periodic real-time tasks in virtualized cloud environment. IEEE Access 7, 29533–29542 (2019)CrossRef
43.
Zurück zum Zitat Aman, M., Said, A.B.M., Kadir, S.J.A., Ullah, I.: Key concept identification: a sentence parse tree-based technique for candidate feature extraction from unstructured texts. IEEE Access 6, 60403–60413 (2018)CrossRef Aman, M., Said, A.B.M., Kadir, S.J.A., Ullah, I.: Key concept identification: a sentence parse tree-based technique for candidate feature extraction from unstructured texts. IEEE Access 6, 60403–60413 (2018)CrossRef
44.
Zurück zum Zitat Abdulkadir, S.J., Yong, S.-P.: Lorenz time-series analysis using a scaled hybrid model. In: 2015 International Symposium on Mathematical Sciences and Computing Research (iSMSC). IEEE, pp. 373–378 (2015) Abdulkadir, S.J., Yong, S.-P.: Lorenz time-series analysis using a scaled hybrid model. In: 2015 International Symposium on Mathematical Sciences and Computing Research (iSMSC). IEEE, pp. 373–378 (2015)
Metadaten
Titel
Hyperparameter Optimization of Evolving Spiking Neural Network for Time-Series Classification
verfasst von
Tasbiha Ibad
Said Jadid Abdulkadir
Norshakirah Aziz
Mohammed Gamal Ragab
Qasem Al-Tashi
Publikationsdatum
24.03.2022
Verlag
Ohmsha
Erschienen in
New Generation Computing / Ausgabe 1/2022
Print ISSN: 0288-3635
Elektronische ISSN: 1882-7055
DOI
https://doi.org/10.1007/s00354-022-00165-3

Weitere Artikel der Ausgabe 1/2022

New Generation Computing 1/2022 Zur Ausgabe

Premium Partner