Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 8/2018

03-03-2017 | Original Article

Two swarm intelligence approaches for tuning extreme learning machine

Authors: Abobakr Khalil Alshamiri, Alok Singh, Bapi Raju Surampudi

Published in: International Journal of Machine Learning and Cybernetics | Issue 8/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Extreme learning machine (ELM) is a new algorithm for training single-hidden layer feedforward neural networks which provides good performance as well as fast learning speed. ELM tends to produce good generalization performance with large number of hidden neurons as the input weights and hidden neurons biases are randomly initialized and remain unchanged during the learning process, and the output weights are analytically determined. In this paper, two swarm intelligence based metaheuristic techniques, viz. Artificial Bee Colony (ABC) and Invasive Weed Optimization (IWO) are proposed for tuning the input weights and hidden biases. The proposed approaches are called ABC-ELM and IWO-ELM in which the input weights and hidden biases are selected using ABC and IWO respectively and the output weights are computed using the Moore-Penrose (MP) generalized inverse. The proposed approaches are tested on different benchmark classification data sets and simulations show that the proposed approaches obtain good generalization performance in comparison to the other techniques available in the literature.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: A new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990 Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: A new learning scheme of feedforward neural networks. In: Proceedings of International Joint Conference on Neural Networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990
2.
go back to reference Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef
3.
go back to reference Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef
4.
go back to reference Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef
5.
go back to reference Cao J, Chen T, Fan J (2016) Landmark recognition with compact bow histogram and ensemble elm. Multimed Tools Appl 75(5):2839–2857CrossRef Cao J, Chen T, Fan J (2016) Landmark recognition with compact bow histogram and ensemble elm. Multimed Tools Appl 75(5):2839–2857CrossRef
6.
go back to reference Cao J, Zhang K, Luo M, Yin C, Lai X (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102CrossRef Cao J, Zhang K, Luo M, Yin C, Lai X (2016) Extreme learning machine and adaptive sparse representation for image classification. Neural Netw 81:91–102CrossRef
7.
go back to reference Serre D (2002) Matrices: theory and applications. Springer, NewYorkMATH Serre D (2002) Matrices: theory and applications. Springer, NewYorkMATH
8.
go back to reference Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 42(2):513–529CrossRef Huang GB, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern B Cybern 42(2):513–529CrossRef
9.
go back to reference Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72:359–366CrossRef Rong HJ, Ong YS, Tan AH, Zhu Z (2008) A fast pruned-extreme learning machine for classification problem. Neurocomputing 72:359–366CrossRef
10.
go back to reference Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRef Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRef
11.
go back to reference Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3191–3199CrossRef Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3191–3199CrossRef
12.
go back to reference Zhu QY, Qin A, Suganthan P, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recogn 38:1759–1763CrossRefMATH Zhu QY, Qin A, Suganthan P, Huang GB (2005) Evolutionary extreme learning machine. Pattern Recogn 38:1759–1763CrossRefMATH
13.
go back to reference Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36:285–305CrossRef Cao J, Lin Z, Huang GB (2012) Self-adaptive evolutionary extreme learning machine. Neural Process Lett 36:285–305CrossRef
14.
go back to reference Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. technical report tr06, Computer Engineering Department. Erciyes University, Turkey Karaboga D (2005) An idea based on honey bee swarm for numerical optimization. technical report tr06, Computer Engineering Department. Erciyes University, Turkey
15.
go back to reference Mehrabiana A, Lucas C (2006) A novel numerical optimization algorithm inspired from weed colonization. Ecol Inform 1:355–366CrossRef Mehrabiana A, Lucas C (2006) A novel numerical optimization algorithm inspired from weed colonization. Ecol Inform 1:355–366CrossRef
16.
go back to reference Akay B, Karaboga D (2012) A modified artificial bee colony algorithm for real-parameter optimization. Inf Sci 192:120–142CrossRef Akay B, Karaboga D (2012) A modified artificial bee colony algorithm for real-parameter optimization. Inf Sci 192:120–142CrossRef
17.
go back to reference Basak A, Pal S, Das S, Abraham A, Snasel V (2010) A modified invasive weed optimization algorithm for time-modulated linear antenna array synthesis. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC). Barcelona, pp 1–8 Basak A, Pal S, Das S, Abraham A, Snasel V (2010) A modified invasive weed optimization algorithm for time-modulated linear antenna array synthesis. In: Proceedings of IEEE Congress on Evolutionary Computation (CEC). Barcelona, pp 1–8
18.
go back to reference Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2(3):321–355MathSciNetMATH Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2(3):321–355MathSciNetMATH
19.
go back to reference Schmidt WF, Kraaijveld MA, Duin RP (1992) Feedforward neural networks with random weights. In: Proceedings of 11th IAPR International Conference on Pattern Recognition Methodology and Systems. Hague, Netherlands, pp 1–4 Schmidt WF, Kraaijveld MA, Duin RP (1992) Feedforward neural networks with random weights. In: Proceedings of 11th IAPR International Conference on Pattern Recognition Methodology and Systems. Hague, Netherlands, pp 1–4
20.
go back to reference Pao YH, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of random vector functional-link net. Neurocomputing 6:163–180CrossRef Pao YH, Park GH, Sobajic DJ (1994) Learning and generalization characteristics of random vector functional-link net. Neurocomputing 6:163–180CrossRef
21.
go back to reference Huang GB (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281CrossRef Huang GB (2003) Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans Neural Netw 14(2):274–281CrossRef
22.
go back to reference Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39(3):459–471MathSciNetCrossRefMATH Karaboga D, Basturk B (2007) A powerful and efficient algorithm for numerical function optimization: artificial bee colony (ABC) algorithm. J Glob Optim 39(3):459–471MathSciNetCrossRefMATH
23.
go back to reference Pandiri V, Singh A (2015) Two metaheuristic approaches for the multiple traveling salesperson problem. Appl Soft Comput 26:74–89CrossRef Pandiri V, Singh A (2015) Two metaheuristic approaches for the multiple traveling salesperson problem. Appl Soft Comput 26:74–89CrossRef
24.
go back to reference Zhou Y, Luo Q, Chen H, He A, Wu J (2015) A discrete invasive weed optimization algorithm for solving traveling salesman problem. Neurocomputing 151:1227–1236CrossRef Zhou Y, Luo Q, Chen H, He A, Wu J (2015) A discrete invasive weed optimization algorithm for solving traveling salesman problem. Neurocomputing 151:1227–1236CrossRef
25.
go back to reference Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7:179–188CrossRef Fisher RA (1936) The use of multiple measurements in taxonomic problems. Ann Eugen 7:179–188CrossRef
26.
go back to reference Lu S, Wang XZ, Zhanga G, Zhoua X (2015) Effective algorithms of the moore-penrose inverse matrices for extreme learning machine. Intell Data Anal 19:743–760CrossRef Lu S, Wang XZ, Zhanga G, Zhoua X (2015) Effective algorithms of the moore-penrose inverse matrices for extreme learning machine. Intell Data Anal 19:743–760CrossRef
27.
go back to reference Ashfaq RAR, Wang XZ, Huang JZ, Abbas H, He YL (2017) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497CrossRef Ashfaq RAR, Wang XZ, Huang JZ, Abbas H, He YL (2017) Fuzziness based semi-supervised learning approach for intrusion detection system. Inf Sci 378:484–497CrossRef
28.
go back to reference Wang XZ, Aamir R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29:1185–1196MathSciNetCrossRef Wang XZ, Aamir R, Fu AM (2015) Fuzziness based sample categorization for classifier performance improvement. J Intell Fuzzy Syst 29:1185–1196MathSciNetCrossRef
29.
go back to reference Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23:1638–1654CrossRef Wang XZ, Xing HJ, Li Y, Hua Q, Dong CR, Pedrycz W (2015) A study on relationship between generalization abilities and fuzziness of base classifiers in ensemble learning. IEEE Trans Fuzzy Syst 23:1638–1654CrossRef
Metadata
Title
Two swarm intelligence approaches for tuning extreme learning machine
Authors
Abobakr Khalil Alshamiri
Alok Singh
Bapi Raju Surampudi
Publication date
03-03-2017
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 8/2018
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-017-0642-3

Other articles of this Issue 8/2018

International Journal of Machine Learning and Cybernetics 8/2018 Go to the issue