Skip to main content
Top
Published in: Neural Processing Letters 1/2018

01-11-2017

A Deep Neural Network Based on ELM for Semi-supervised Learning of Image Classification

Authors: Peiju Chang, Jiangshe Zhang, Junying Hu, Zengjie Song

Published in: Neural Processing Letters | Issue 1/2018

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Deep learning has become one of very important machine learning methods in image classification, but most of them require a long training time to solve a non-convex optimization problem. In comparison, the training of extreme learning machine (ELM) is very simple, fast and effective. In order to combine the advantages of both methods, many researchers have tried to introduce ELM to deep architectures (Kasun et al. in IEEE Intell Syst 28:31–34, 2013; Yu et al. in Neurocomputing 149:308–315, 2015; Tissera and McDonnell in Neurocomputing 174:42–49, 2016 and in: Proceedings of ELM-2014, vol 1, Proceedings in adaptation, learning and optimization, vol 3, 2016; Junying et al. in Neurocomputing 171:63–72, 2016; Uzair et al. in Neural Comput Appl, 2015) to solve unsupervised learning and supervised learning problems. In this paper, we propose a new deep neural network based on ELM called discriminative deep ELM (DDELM) to address the semi-supervised learning problems in image classification. The proposed deep architecture consists of several stacked unsupervised ELMs and an additional label layer on the top layer of the stacked model. Experiments on three standard image data show that DDELM outperforms both representative semi-supervised learning algorithms and existing deep architectures such as DCNN in terms of accuracy and training time.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Huang GB, Zhu QY, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE International joint conference on neural networks 2:985–990 Huang GB, Zhu QY, Siew C-K (2004) Extreme learning machine: a new learning scheme of feedforward neural networks. IEEE International joint conference on neural networks 2:985–990
3.
go back to reference Kasun LLC, Zhou H, Huang GB (2013) Representational learning with ELMs for big data. IEEE Intell Syst 28:31–34CrossRef Kasun LLC, Zhou H, Huang GB (2013) Representational learning with ELMs for big data. IEEE Intell Syst 28:31–34CrossRef
4.
go back to reference Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRef Yu W, Zhuang F, He Q, Shi Z (2015) Learning deep representations via extreme learning machines. Neurocomputing 149:308–315CrossRef
5.
go back to reference Tissera MD, McDonnell MD (2016) Deep extreme learning machines: supervised autoencoding architecture for classification. Neurocomputing 174:42–49CrossRef Tissera MD, McDonnell MD (2016) Deep extreme learning machines: supervised autoencoding architecture for classification. Neurocomputing 174:42–49CrossRef
6.
go back to reference Tissera MD, McDonnell MD (2014) Deep extreme learning machines for classification. In: Proceedings of ELM-2014, vol 1, Proceedings in adaptation, learning and optimization, vol 3. pp 345–354 Tissera MD, McDonnell MD (2014) Deep extreme learning machines for classification. In: Proceedings of ELM-2014, vol 1, Proceedings in adaptation, learning and optimization, vol 3. pp 345–354
7.
go back to reference Junying H, Jiangshe Z, Chunxia Z et al (2016) A new deep neural network based on a stack of single-hidden-layer feedforward neural networks with randomly fixed hidden neurons. Neurocomputing 171:63–72CrossRef Junying H, Jiangshe Z, Chunxia Z et al (2016) A new deep neural network based on a stack of single-hidden-layer feedforward neural networks with randomly fixed hidden neurons. Neurocomputing 171:63–72CrossRef
8.
go back to reference Uzair M, Shafait F, Ghanem B, Mian A (2015) Representation learning with deep extreme learning machines for efficient image set classification. Neural Comput Appl Uzair M, Shafait F, Ghanem B, Mian A (2015) Representation learning with deep extreme learning machines for efficient image set classification. Neural Comput Appl
9.
go back to reference Liu Y, Zhou S et al (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44:2287–2296CrossRefMATH Liu Y, Zhou S et al (2011) Discriminative deep belief networks for visual data classification. Pattern Recognit 44:2287–2296CrossRefMATH
11.
go back to reference Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. In: NIPS Bengio Y, Lamblin P, Popovici D, Larochelle H (2006) Greedy layer-wise training of deep networks. In: NIPS
12.
go back to reference Huang G, Song SJ et al (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern Extreme Learn Mach 44:2168–2267 Huang G, Song SJ et al (2014) Semi-supervised and unsupervised extreme learning machines. IEEE Trans Cybern Extreme Learn Mach 44:2168–2267
13.
go back to reference Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, CambridgeCrossRef Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MIT Press, CambridgeCrossRef
14.
go back to reference Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MITPress, CambridgeCrossRef Chapelle O, Scholkopf B, Zien A (2006) Semi-supervised learning. MITPress, CambridgeCrossRef
15.
go back to reference Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection models. In: Seventh IEEE workshops on application of computer vision. pp 29–36 Rosenberg C, Hebert M, Schneiderman H (2005) Semi-supervised self-training of object detection models. In: Seventh IEEE workshops on application of computer vision. pp 29–36
16.
go back to reference Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. International workshop on artificial intelligence and statistics 1:57–64 Chapelle O, Zien A (2005) Semi-supervised classification by low density separation. International workshop on artificial intelligence and statistics 1:57–64
17.
go back to reference Sindhwani V, Niyogi P, Belkin M (2005) Beyond the point cloud: from transductive to semi-supervised learning. International conference on machine learning, ACM, Bonn, Germany 22:824–831 Sindhwani V, Niyogi P, Belkin M (2005) Beyond the point cloud: from transductive to semi-supervised learning. International conference on machine learning, ACM, Bonn, Germany 22:824–831
18.
go back to reference Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712MathSciNetMATH Collobert R, Sinz F, Weston J, Bottou L (2006) Large scale transductive SVMs. J Mach Learn Res 7:1687–1712MathSciNetMATH
19.
go back to reference Blum A, Lafferty J Rwebangira MR et al (2004) Semi-supervised learning using randomized mincuts. In: Proceedings of the international conference on machine learning (ICML) Blum A, Lafferty J Rwebangira MR et al (2004) Semi-supervised learning using randomized mincuts. In: Proceedings of the international conference on machine learning (ICML)
20.
go back to reference Zhu X, Ghahramani Z et al (2003) Semi-supervised learning using Gaussian fields and harmonic functions. Proceddings of the international conference on machine learning (ICML) 3:912–919 Zhu X, Ghahramani Z et al (2003) Semi-supervised learning using Gaussian fields and harmonic functions. Proceddings of the international conference on machine learning (ICML) 3:912–919
21.
go back to reference Fergus R, Weiss Y, Torralba A (2009) Semi-supervised learning in gigantic image collections. In: Advances in neural information processing systems (NIPS) Fergus R, Weiss Y, Torralba A (2009) Semi-supervised learning in gigantic image collections. In: Advances in neural information processing systems (NIPS)
22.
go back to reference Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. International conference on machine learning. ACM, Helsinki, pp 1168–1175 Weston J, Ratle F, Collobert R (2008) Deep learning via semi-supervised embedding. International conference on machine learning. ACM, Helsinki, pp 1168–1175
23.
go back to reference Zhu X (2007) Semi-supervised learning literature survey. Technical report, University of Wisconsin Madison, Madison, 123 Zhu X (2007) Semi-supervised learning literature survey. Technical report, University of Wisconsin Madison, Madison, 123
24.
go back to reference Salakhutdinov RR, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. In: Proceedings of eleventh international conference on artificial intelligence and statistics Salakhutdinov RR, Hinton GE (2007) Learning a nonlinear embedding by preserving class neighbourhood structure. In: Proceedings of eleventh international conference on artificial intelligence and statistics
25.
go back to reference Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: COLT Boser BE, Guyon IM, Vapnik VN (1992) A training algorithm for optimal margin classifiers. In: COLT
26.
go back to reference Jarrett K, Kavukcuoglu K, Ranzato M, Cun YL (2009) What is the best multi-stage architecture for object recognition. In: ICCV Jarrett K, Kavukcuoglu K, Ranzato M, Cun YL (2009) What is the best multi-stage architecture for object recognition. In: ICCV
27.
go back to reference Li FF, Fergus R, Pernoa P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR Li FF, Fergus R, Pernoa P (2004) Learning generative visual models from few training examples: an incremental Bayesian approach tested on 101 object categories. In: CVPR
28.
go back to reference Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. In: IJCV Oliva A, Torralba A (2001) Modeling the shape of the scene: a holistic representation of the spatial envelope. In: IJCV
29.
go back to reference Pronobis A, Caputo B, Jensfelt P, Christensen HI (2010) A realistic benchmark for visual indoor place recognition. Robot Auton Syst 58:81–96CrossRef Pronobis A, Caputo B, Jensfelt P, Christensen HI (2010) A realistic benchmark for visual indoor place recognition. Robot Auton Syst 58:81–96CrossRef
30.
go back to reference Zhong S, Liu Y, Yang Liu (2011) Bilinear deep learning for image classification. In: ACM conference on multimedia. pp 343–352 Zhong S, Liu Y, Yang Liu (2011) Bilinear deep learning for image classification. In: ACM conference on multimedia. pp 343–352
31.
go back to reference Sim T, Baker S (2003) The CMU pose, illumination and expression database. PAMI 25:1615–1618CrossRef Sim T, Baker S (2003) The CMU pose, illumination and expression database. PAMI 25:1615–1618CrossRef
32.
go back to reference He X.F, Cai D, Niyogi P (2005) Tensor subspace analysis. In: NIPS He X.F, Cai D, Niyogi P (2005) Tensor subspace analysis. In: NIPS
33.
Metadata
Title
A Deep Neural Network Based on ELM for Semi-supervised Learning of Image Classification
Authors
Peiju Chang
Jiangshe Zhang
Junying Hu
Zengjie Song
Publication date
01-11-2017
Publisher
Springer US
Published in
Neural Processing Letters / Issue 1/2018
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-017-9709-0

Other articles of this Issue 1/2018

Neural Processing Letters 1/2018 Go to the issue