Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 5/2016

01.10.2016 | Original Article

On optimization based extreme learning machine in primal for regression and classification by functional iterative method

verfasst von: S. Balasundaram, Deepak Gupta

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 5/2016

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, the recently proposed extreme learning machine in the aspect of optimization method by Huang et al. (Neurocomputing, 74: 155–163, 2010) has been considered in its primal form whose solution is obtained by solving an absolute value equation problem by a simple, functional iterative algorithm. It has been proved under sufficient conditions that the algorithm converges linearly. The pseudo codes of the algorithm for regression and classification are given and they can be easily implemented in MATLAB. Experiments were performed on a number of real-world datasets using additive and radial basis function hidden nodes. Similar or better generalization performance of the proposed method in comparison to support vector machine (SVM), extreme learning machine (ELM), optimally pruned extreme learning machine (OP-ELM) and optimization based extreme learning machine (OB-ELM) methods with faster learning speed than SVM and OB-ELM demonstrates its effectiveness and usefulness.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Balasundaram S, Gupta D, Kapil (2014) 1-norm extreme learning machine for regression and multiclass classification using Newton method. Neuocomputing 128:4–14CrossRef Balasundaram S, Gupta D, Kapil (2014) 1-norm extreme learning machine for regression and multiclass classification using Newton method. Neuocomputing 128:4–14CrossRef
2.
Zurück zum Zitat Balasundaram S, Kapil (2013) On extreme learning machine for ε–insensitive regression in the primal by Newton method. Neural Comput Appl 22:559–567CrossRef Balasundaram S, Kapil (2013) On extreme learning machine for ε–insensitive regression in the primal by Newton method. Neural Comput Appl 22:559–567CrossRef
3.
Zurück zum Zitat Balasundaram S, Kapil (2011) Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives. Neurocomputing 74:2511–2519CrossRef Balasundaram S, Kapil (2011) Application of error minimized extreme learning machine for simultaneous learning of a function and its derivatives. Neurocomputing 74:2511–2519CrossRef
4.
Zurück zum Zitat Binu PC, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161CrossRef Binu PC, Vimal Krishnan VR, Raju G, Babu Anto P (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern 3(2):149–161CrossRef
5.
Zurück zum Zitat Box GEP, Jenkins GM (1976) Time series analysis: forecasting and Control. Holden-Day, San FranciscoMATH Box GEP, Jenkins GM (1976) Time series analysis: forecasting and Control. Holden-Day, San FranciscoMATH
6.
Zurück zum Zitat Chen Z, Zhu H, Wang Y (2013) A modified extreme learning machine with sigmoidal activation functions. Neural Comput Appl 22:541–550MathSciNetCrossRef Chen Z, Zhu H, Wang Y (2013) A modified extreme learning machine with sigmoidal activation functions. Neural Comput Appl 22:541–550MathSciNetCrossRef
7.
Zurück zum Zitat Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel based learning methods. Cambridge University Press, CambridgeCrossRefMATH Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel based learning methods. Cambridge University Press, CambridgeCrossRefMATH
9.
Zurück zum Zitat Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1-30 Demsar J (2006) Statistical comparisons of classifiers over multiple data sets. J Mach Learn Res 7: 1-30
10.
Zurück zum Zitat Deng W, Chen L (2010) Color image watermarking using regularized extreme learning machine. Neural Netw World 20(3):317–330 Deng W, Chen L (2010) Color image watermarking using regularized extreme learning machine. Neural Netw World 20(3):317–330
11.
Zurück zum Zitat Gretton A, Doucet A, Herbrich R, Rayner PJW, Scholkopf B (2001) Support vector regression for black-box system identification, In: Proceedings of the 11th IEEE Workshop on Statistical Signal Processing Gretton A, Doucet A, Herbrich R, Rayner PJW, Scholkopf B (2001) Support vector regression for black-box system identification, In: Proceedings of the 11th IEEE Workshop on Statistical Signal Processing
12.
Zurück zum Zitat Feng G, Huang G-B, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357CrossRef Feng G, Huang G-B, Lin Q, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357CrossRef
13.
Zurück zum Zitat Frenay B, Verleysen M (2010) Using SVMs with randomized feature spaces: an extreme learning approach. In: Proceedings of the 18th European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, pp. 315–320 Frenay B, Verleysen M (2010) Using SVMs with randomized feature spaces: an extreme learning approach. In: Proceedings of the 18th European Symposium on Artificial Neural Networks (ESANN), Bruges, Belgium, pp. 315–320
14.
15.
Zurück zum Zitat Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef Huang G-B, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef
16.
Zurück zum Zitat Huang G-B, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468CrossRef Huang G-B, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468CrossRef
17.
Zurück zum Zitat Huang G-B, Chen L, Siew C-K (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef Huang G-B, Chen L, Siew C-K (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef
18.
Zurück zum Zitat Huang G-B, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRef Huang G-B, Ding X, Zhou H (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRef
19.
Zurück zum Zitat Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122CrossRef Huang G-B, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybern 2:107–122CrossRef
20.
Zurück zum Zitat Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst, Man Cybern Part B Cybern 42(2):513–528CrossRef Huang G-B, Zhou H, Ding X, Zhang R (2012) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst, Man Cybern Part B Cybern 42(2):513–528CrossRef
21.
Zurück zum Zitat Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef Huang G-B, Zhu Q-Y, Siew C-K (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef
22.
Zurück zum Zitat Jun W, Shitong W, Chung F (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271CrossRef Jun W, Shitong W, Chung F (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271CrossRef
23.
Zurück zum Zitat Lan Y, Soh C, Huang G-B (2010) Two-stage extreme learning machine regression. Neurocomputing 73:3028–3038CrossRef Lan Y, Soh C, Huang G-B (2010) Two-stage extreme learning machine regression. Neurocomputing 73:3028–3038CrossRef
24.
Zurück zum Zitat Liu Q, He Q, Shi Z (2008) Extreme support vector machine classifier. LNCS 5012:222–233 Liu Q, He Q, Shi Z (2008) Extreme support vector machine classifier. LNCS 5012:222–233
26.
Zurück zum Zitat Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRef Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):158–162CrossRef
27.
Zurück zum Zitat Minhas R, Baradarani A, Seifzadeh S, Wu QMJ (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917CrossRef Minhas R, Baradarani A, Seifzadeh S, Wu QMJ (2010) Human action recognition using extreme learning machine based on visual vocabularies. Neurocomputing 73:1906–1917CrossRef
28.
Zurück zum Zitat Mohammed AA, Wu QMJ, Sid-Ahmed MA (2010) Application of wave atoms decomposition and extreme learning machine for finger print classification. LNCS 6112:246–256 Mohammed AA, Wu QMJ, Sid-Ahmed MA (2010) Application of wave atoms decomposition and extreme learning machine for finger print classification. LNCS 6112:246–256
29.
Zurück zum Zitat Mukherjee S, Osuna E, Girosi F (1997) Nonlinear prediction of chaotic time series using support vector machines, in: NNSP’97: Neural Networks for Signal Processing VII: in Proceedigs of IEEE Signal Processing Society Workshop, Amelia Island, FL, USA, pp. 511–520 Mukherjee S, Osuna E, Girosi F (1997) Nonlinear prediction of chaotic time series using support vector machines, in: NNSP’97: Neural Networks for Signal Processing VII: in Proceedigs of IEEE Signal Processing Society Workshop, Amelia Island, FL, USA, pp. 511–520
30.
Zurück zum Zitat Muller KR, Smola AJ, Ratsch G, Schölkopf B, Kohlmorgen J (1999) Using support vector machines for time series prediction. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 243–254 Muller KR, Smola AJ, Ratsch G, Schölkopf B, Kohlmorgen J (1999) Using support vector machines for time series prediction. In: Schölkopf B, Burges CJC, Smola AJ (eds) Advances in kernel methods—support vector learning. MIT Press, Cambridge, pp 243–254
32.
Zurück zum Zitat Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New YorkMATH Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New YorkMATH
33.
Zurück zum Zitat Samet S, Miri A (2012) Privacy-preserving back-propagation and extreme learning machine algorithms. Data Knowl Eng 79–80:40–61CrossRef Samet S, Miri A (2012) Privacy-preserving back-propagation and extreme learning machine algorithms. Data Knowl Eng 79–80:40–61CrossRef
34.
Zurück zum Zitat Singh R, Balasundaram S (2007) Application of extreme learning machine for time series analysis. Int J Intell Technol 2:256–262 Singh R, Balasundaram S (2007) Application of extreme learning machine for time series analysis. Int J Intell Technol 2:256–262
35.
Zurück zum Zitat Sjoberg J, Zhang Q, Ljung L, Berveniste A, Delyon B, Glorennec P, Hjalmarsson H, Juditsky (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31:1691–1724MathSciNetCrossRefMATH Sjoberg J, Zhang Q, Ljung L, Berveniste A, Delyon B, Glorennec P, Hjalmarsson H, Juditsky (1995) Nonlinear black-box modeling in system identification: a unified overview. Automatica 31:1691–1724MathSciNetCrossRefMATH
36.
Zurück zum Zitat Souza LGM, Barreto GA (2006) Nonlinear system identification using local ARX models based on the self-organizing map. Learning and Nonlinear Models-Revista da Sociedade Brasileira de Redes Neurals (SBRN) 4(2):112–123CrossRef Souza LGM, Barreto GA (2006) Nonlinear system identification using local ARX models based on the self-organizing map. Learning and Nonlinear Models-Revista da Sociedade Brasileira de Redes Neurals (SBRN) 4(2):112–123CrossRef
37.
Zurück zum Zitat Vapnik VN (2000) The nature of statistical learning theory, 2nd edn. Springer, New YorkCrossRefMATH Vapnik VN (2000) The nature of statistical learning theory, 2nd edn. Springer, New YorkCrossRefMATH
38.
Zurück zum Zitat Wang X, Shao Q, Qing M, Zhai J (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9CrossRef Wang X, Shao Q, Qing M, Zhai J (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3–9CrossRef
39.
Zurück zum Zitat Wang X, Chen A, Feng H (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef Wang X, Chen A, Feng H (2011) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef
40.
Zurück zum Zitat Wang XX, Chen S, Lowe D, Harris CJ (2006) Sparse support vector regression based on orthogonal forward selection for generalized kernel model. Neurocomputing 70:462–474CrossRef Wang XX, Chen S, Lowe D, Harris CJ (2006) Sparse support vector regression based on orthogonal forward selection for generalized kernel model. Neurocomputing 70:462–474CrossRef
41.
Zurück zum Zitat Yuan Y, Wang Y, Cao F (2011) Optimization approximation solution for regression problem based on extreme learning machine. Neurocomputing 74:2475–2482CrossRef Yuan Y, Wang Y, Cao F (2011) Optimization approximation solution for regression problem based on extreme learning machine. Neurocomputing 74:2475–2482CrossRef
42.
Zurück zum Zitat Zhai J, Xu H, Wang X (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502CrossRef Zhai J, Xu H, Wang X (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493–1502CrossRef
43.
Zurück zum Zitat Zhou GL, Toh KC (2005) Super linear convergence of a newton-type algorithm for monotone equations. J Optim Theory Appl 125:205–221MathSciNetCrossRefMATH Zhou GL, Toh KC (2005) Super linear convergence of a newton-type algorithm for monotone equations. J Optim Theory Appl 125:205–221MathSciNetCrossRefMATH
44.
Zurück zum Zitat Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recogn 38:1759–1763CrossRefMATH Zhu Q-Y, Qin AK, Suganthan PN, Huang G-B (2005) Evolutionary extreme learning machine. Pattern Recogn 38:1759–1763CrossRefMATH
Metadaten
Titel
On optimization based extreme learning machine in primal for regression and classification by functional iterative method
verfasst von
S. Balasundaram
Deepak Gupta
Publikationsdatum
01.10.2016
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 5/2016
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-014-0283-8

Weitere Artikel der Ausgabe 5/2016

International Journal of Machine Learning and Cybernetics 5/2016 Zur Ausgabe

Neuer Inhalt