Skip to main content
Top
Published in: Soft Computing 9/2012

01-09-2012 | Focus

Generalized extreme learning machine acting on a metric space

Authors: Jianwei Zhao, Dong Sun Park, Joonwhoan Lee, Feilong Cao

Published in: Soft Computing | Issue 9/2012

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Functional data learning is an extension of traditional data learning, that is, learning the data chosen from the Euclidean space \({\mathbb{R}^{n}}\) to a metric space. This paper focuses on the functional data learning with generalized single-hidden layer feedforward neural networks (GSLFNs) acting on some metric spaces. In addition, three learning algorithms, named Hilbert parallel overrelaxation backpropagation (H-PORBP) algorithm, ν-generalized support vector regression (ν-GSVR) and generalized extreme learning machine (G-ELM) are proposed to train the GSLFNs acting on some metric spaces. The experimental results on some metric spaces indicate that GELM with additive/RBF hidden-nodes has a faster learning speed, a better accuracy, and a better stability than HPORBP algorithm and ν-GSVR for training the functional data. The idea of GELM can be used to extend those improved extreme learning machines (ELMs) that act on the Euclidean space \({\mathbb{R}^{n}, }\) such as online sequential ELM, incremental ELM, pruning ELM and so on, to some metric spaces.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Cao FL, Zhang R (2009) The errors of approximation for feedforward neural networks in the L p metric.. Math Comput Model 49(7–8):1563–1572MathSciNetMATHCrossRef Cao FL, Zhang R (2009) The errors of approximation for feedforward neural networks in the L p metric.. Math Comput Model 49(7–8):1563–1572MathSciNetMATHCrossRef
go back to reference Cao FL, Zhang YQ, He ZR (2009) Interpolation and rate of convergence for a class of neural networks.. Appl Math Model 33(3):1441–1456MathSciNetMATHCrossRef Cao FL, Zhang YQ, He ZR (2009) Interpolation and rate of convergence for a class of neural networks.. Appl Math Model 33(3):1441–1456MathSciNetMATHCrossRef
go back to reference Cao FL, Lin SB, Xu ZB (2010) Constructive approximate interpolation by neural networks in the metric space. Math Comput Model 52:1674–1681MathSciNetMATHCrossRef Cao FL, Lin SB, Xu ZB (2010) Constructive approximate interpolation by neural networks in the metric space. Math Comput Model 52:1674–1681MathSciNetMATHCrossRef
go back to reference Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern. doi:10.1007/s13042-011-0049-5 Chacko BP, Vimal Krishnan VR, Raju G, Babu Anto P (2011) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybern. doi:10.​1007/​s13042-011-0049-5
go back to reference Chen TP, Chen H, Liu RW (1995) Approximation capability in by multiplayer feedforward networks and related problems. IEEE Trans Neural Netw 6:25–30MATHCrossRef Chen TP, Chen H, Liu RW (1995) Approximation capability in by multiplayer feedforward networks and related problems. IEEE Trans Neural Netw 6:25–30MATHCrossRef
go back to reference Chen TP, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6:911–917CrossRef Chen TP, Chen H (1995) Universal approximation to nonlinear operators by neural networks with arbitrary activation functions and its application to dynamical systems. IEEE Trans Neural Netw 6:911–917CrossRef
go back to reference Conway JB (1985) A course in functional analysis. GTM96, Springer-Verlag, New York, USA Conway JB (1985) A course in functional analysis. GTM96, Springer-Verlag, New York, USA
go back to reference Corrieu P (2005) Function approximation on non-Euclidean spaces. Neural Netw 18:91–102CrossRef Corrieu P (2005) Function approximation on non-Euclidean spaces. Neural Netw 18:91–102CrossRef
go back to reference Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297MATH Cortes C, Vapnik V (1995) Support vector networks. Mach Learn 20(3):273–297MATH
go back to reference Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74:447–456CrossRef Deng WY, Zheng QH, Lian SG, Chen L, Wang X (2010) Ordinal extreme learning machine. Neurocomputing 74:447–456CrossRef
go back to reference Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357CrossRef Feng GR, Huang GB, Lin QP, Gay R (2009) Error minimized extreme learning machine with growth of hidden nodes and incremental learning. IEEE Trans Neural Netw 20(8):1352–1357CrossRef
go back to reference Fukuoka Y, Matsuki H, Minamitani H (1998) A modified backpropagation method to avoid false local minima. Neural Netw 11:1059–1072CrossRef Fukuoka Y, Matsuki H, Minamitani H (1998) A modified backpropagation method to avoid false local minima. Neural Netw 11:1059–1072CrossRef
go back to reference Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192CrossRef Funahashi KI (1989) On the approximate realization of continuous mappings by neural networks. Neural Netw 2:183–192CrossRef
go back to reference Heeswijk M, Miche Y, Oja E, Lendasse A (2011) GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16):2430–2437CrossRef Heeswijk M, Miche Y, Oja E, Lendasse A (2011) GPU-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16):2430–2437CrossRef
go back to reference Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072CrossRef Hornik K (1993) Some new results on neural network approximation. Neural Netw 6:1069–1072CrossRef
go back to reference Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990 Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. In: Proceedings of international joint conference on neural networks (IJCNN), vol 2, Budapest, Hungary, pp 985–990
go back to reference Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70:489–501CrossRef
go back to reference Huang GB, Zhu QY, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef Huang GB, Zhu QY, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892CrossRef
go back to reference Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef Huang GB, Chen L (2007) Convex incremental extreme learning machine. Neurocomputing 70:3056–3062CrossRef
go back to reference Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468CrossRef Huang GB, Chen L (2008) Enhanced random search based incremental extreme learning machine. Neurocomputing 71:3460–3468CrossRef
go back to reference Huang GB, Ding X, Zhou X (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRef Huang GB, Ding X, Zhou X (2010) Optimization method based extreme learning machine for classification. Neurocomputing 74:155–163CrossRef
go back to reference Huang GB, Wang DH, Lan Y (2011) Extreme learning machine: a survey. Int J Mach Learn Cybern 2(2):107–122CrossRef Huang GB, Wang DH, Lan Y (2011) Extreme learning machine: a survey. Int J Mach Learn Cybern 2(2):107–122CrossRef
go back to reference Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5:954–975CrossRef Kadirkamanathan V, Niranjan M (1993) A function estimation approach to sequential learning with neural networks. Neural Comput 5:954–975CrossRef
go back to reference Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395CrossRef Lan Y, Soh YC, Huang GB (2009) Ensemble of online sequential extreme learning machine. Neurocomputing 72:3391–3395CrossRef
go back to reference Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3193–3199 Lan Y, Soh YC, Huang GB (2010) Constructive hidden nodes selection of extreme learning machine for regression. Neurocomputing 73:3193–3199
go back to reference Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73:3028–3038CrossRef Lan Y, Soh YC, Huang GB (2010) Two-stage extreme learning machine for regression. Neurocomputing 73:3028–3038CrossRef
go back to reference Li J, Han G, Wen J, Gao XB (2011) Robust tensor subspace learning for anomaly detection. Int J Mach Learn Cybern 2(2):89–98CrossRef Li J, Han G, Wen J, Gao XB (2011) Robust tensor subspace learning for anomaly detection. Int J Mach Learn Cybern 2(2):89–98CrossRef
go back to reference Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423CrossRef Liang NY, Huang GB, Saratchandran P, Sundararajan N (2006) A fast and accurate online sequential learning algorithm for feedforward networks. IEEE Trans Neural Netw 17(6):1411–1423CrossRef
go back to reference Mangasarian OL, Solodov MV (1994) Serial and parallel backpropagation convergence via nonmonotone perturbed minimization. Optim method Softw 4(2):103–116CrossRef Mangasarian OL, Solodov MV (1994) Serial and parallel backpropagation convergence via nonmonotone perturbed minimization. Optim method Softw 4(2):103–116CrossRef
go back to reference Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):1411–1423 Miche Y, Sorjamaa A, Bas P, Simula O, Jutten C, Lendasse A (2010) OP-ELM: optimally pruned extreme learning machine. IEEE Trans Neural Netw 21(1):1411–1423
go back to reference Miche Y, Heeswijk M, Bas P, Simula O, Lendasse A (2011) TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16):2413–2421CrossRef Miche Y, Heeswijk M, Bas P, Simula O, Lendasse A (2011) TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16):2413–2421CrossRef
go back to reference Muńoz A, Gonźlez J (2010) Representing functional data using support vector machines. Pattern Recog Lett 31:511–516CrossRef Muńoz A, Gonźlez J (2010) Representing functional data using support vector machines. Pattern Recog Lett 31:511–516CrossRef
go back to reference Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257CrossRef Park J, Sandberg IW (1991) Universal approximation using radial-basis-function networks. Neural Comput 3(2):246–257CrossRef
go back to reference Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New YorkMATH Rao CR, Mitra SK (1971) Generalized inverse of matrices and its applications. Wiley, New YorkMATH
go back to reference Rossi F, Delannay N, Guez BC, Verleysen M (2005) Representation of functional data in neural networks. Neurocomputing 64:83–210CrossRef Rossi F, Delannay N, Guez BC, Verleysen M (2005) Representation of functional data in neural networks. Neurocomputing 64:83–210CrossRef
go back to reference Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagation errors. Nature 323:533–536CrossRef Rumelhart DE, Hinton GE, Williams RJ (1986) Learning representations by backpropagation errors. Nature 323:533–536CrossRef
go back to reference Schölkopf B, Mika S, Burges CJC, Knirsch P, Müller KR, Rätsch G (1999) Input space vs. feature space in kernel-basedmethods. IEEE Trans Neural Netw 10(5):1000–1017 Schölkopf B, Mika S, Burges CJC, Knirsch P, Müller KR, Rätsch G (1999) Input space vs. feature space in kernel-basedmethods. IEEE Trans Neural Netw 10(5):1000–1017
go back to reference Schölkopf B, Smola AJ, Williamson R, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRef Schölkopf B, Smola AJ, Williamson R, Bartlett PL (2000) New support vector algorithms. Neural Comput 12(5):1207–1245CrossRef
go back to reference Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH Vapnik V (1998) Statistical learning theory. Wiley, New YorkMATH
go back to reference Wang XZ, Dong CR (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567CrossRef Wang XZ, Dong CR (2009) Improving generalization of fuzzy if-then rules by maximizing fuzzy entropy. IEEE Trans Fuzzy Syst 17(3):556–567CrossRef
go back to reference Wang XZ, Chen AX, Feng HM (2011a) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef Wang XZ, Chen AX, Feng HM (2011a) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520–2525CrossRef
go back to reference Wang YG, Cao FL, Yuan YB (2011b) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490CrossRef Wang YG, Cao FL, Yuan YB (2011b) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483–2490CrossRef
go back to reference Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271CrossRef Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybern 2(4):261–271CrossRef
go back to reference Wu W, Wang J, Cheng MS, Li ZX (2011) Convergence analysis of online gradient method for BP neural networks. Neural Netw 24:91–98MATHCrossRef Wu W, Wang J, Cheng MS, Li ZX (2011) Convergence analysis of online gradient method for BP neural networks. Neural Netw 24:91–98MATHCrossRef
go back to reference Yamamuro S (1970) Differential calculus in topological linear spaces. Lect Notes Math, vol 374, Springer, Berlin Yamamuro S (1970) Differential calculus in topological linear spaces. Lect Notes Math, vol 374, Springer, Berlin
Metadata
Title
Generalized extreme learning machine acting on a metric space
Authors
Jianwei Zhao
Dong Sun Park
Joonwhoan Lee
Feilong Cao
Publication date
01-09-2012
Publisher
Springer-Verlag
Published in
Soft Computing / Issue 9/2012
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-012-0825-5

Other articles of this Issue 9/2012

Soft Computing 9/2012 Go to the issue

Premium Partner