Skip to main content
Erschienen in: Neural Processing Letters 3/2019

11.06.2018

Fast-Convergent Fully Connected Deep Learning Model Using Constrained Nodes Input

verfasst von: Chen Ding, Ying Li, Lei Zhang, Jinyang Zhang, Lu Yang, Wei Wei, Yong Xia, Yanning Zhang

Erschienen in: Neural Processing Letters | Ausgabe 3/2019

Einloggen

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recently, deep learning models exhibit promising performance in various applications. However, most of them converge slowly due to gradient vanishing. To address this problem, we propose a fast convergent fully connected deep learning network in this study. Through constraining the input values of nodes on the fully connected layers, the proposed method is able to well mitigate the gradient vanishing problems in training phase, and thus greatly reduces the training iterations required to reach convergence. Nevertheless, the drop of generalization performance is negligible. Experimental results validate the effectiveness of the proposed method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
3.
Zurück zum Zitat Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:​1409.​1556
4.
Zurück zum Zitat Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going Deeper with Convolutions, arXiv preprint arXiv:1409.4842 Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going Deeper with Convolutions, arXiv preprint arXiv:​1409.​4842
5.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 He K, Zhang X, Ren S, Sun J (2016) In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778
6.
Zurück zum Zitat Yang W, Zhang H, Yang J, Wu J, Yin X, Chen Y, Shu H, Luo L, Coatrieux G, Gui Z (2017) Improving low-dose ct image using residual convolutional network. IEEE Access 5:24698CrossRef Yang W, Zhang H, Yang J, Wu J, Yin X, Chen Y, Shu H, Luo L, Coatrieux G, Gui Z (2017) Improving low-dose ct image using residual convolutional network. IEEE Access 5:24698CrossRef
8.
Zurück zum Zitat Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2014) Pcanet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017MathSciNetCrossRefMATH Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2014) Pcanet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017MathSciNetCrossRefMATH
9.
Zurück zum Zitat Zeng R, Wu J, Shao Z, Chen Y, Chen B, Senhadji L, Shu H (2016) Color image classification via quaternion principal component analysis network. Neurocomputing 216:416CrossRef Zeng R, Wu J, Shao Z, Chen Y, Chen B, Senhadji L, Shu H (2016) Color image classification via quaternion principal component analysis network. Neurocomputing 216:416CrossRef
10.
Zurück zum Zitat Ding C, Li Y, Xia Y, Wei W, Zhang L, Zhang Y (2017) Convolutional neural networks based hyperspectral image classification method with adaptive kernels. Remote Sens 9(6):618CrossRef Ding C, Li Y, Xia Y, Wei W, Zhang L, Zhang Y (2017) Convolutional neural networks based hyperspectral image classification method with adaptive kernels. Remote Sens 9(6):618CrossRef
12.
Zurück zum Zitat Wei W, Zhang L, Tian C, Plaza A, Zhang Y (2017) Structured sparse coding-based hyperspectral imagery denoising with intracluster filtering. IEEE Trans on Geosci Remote Sens 55(12):6860CrossRef Wei W, Zhang L, Tian C, Plaza A, Zhang Y (2017) Structured sparse coding-based hyperspectral imagery denoising with intracluster filtering. IEEE Trans on Geosci Remote Sens 55(12):6860CrossRef
13.
Zurück zum Zitat Wang C, Zhang L, Wei W, Zhang Y (2018) When low rank representation based hyperspectral imagery classification meets segmented stacked denoising auto-encoder based spatial-spectral feature. Remote Sens 10(2):284CrossRef Wang C, Zhang L, Wei W, Zhang Y (2018) When low rank representation based hyperspectral imagery classification meets segmented stacked denoising auto-encoder based spatial-spectral feature. Remote Sens 10(2):284CrossRef
14.
Zurück zum Zitat Krger J, Westermann R (2003) In: ACM SIGGRAPH, pp 908–916 Krger J, Westermann R (2003) In: ACM SIGGRAPH, pp 908–916
15.
Zurück zum Zitat Byong-Heon K, Burm-Suk S (2005) Design and implementation of jpeg image display board using FFGA. J Digit Contents Soc 6(3):169–174 Byong-Heon K, Burm-Suk S (2005) Design and implementation of jpeg image display board using FFGA. J Digit Contents Soc 6(3):169–174
16.
Zurück zum Zitat Le QV, Ngiam J, Coates A, Lahiri A, Prochnow B, Ng AY (2011) In: International conference on machine learning, ICML 2011, Bellevue, Washington, Usa, June 28–July, pp. 265–272 Le QV, Ngiam J, Coates A, Lahiri A, Prochnow B, Ng AY (2011) In: International conference on machine learning, ICML 2011, Bellevue, Washington, Usa, June 28–July, pp. 265–272
17.
Zurück zum Zitat Orr GB, Müller KR (1998) Neural networks: tricks of the trade. Can J Anaesth 41(7):658 Orr GB, Müller KR (1998) Neural networks: tricks of the trade. Can J Anaesth 41(7):658
18.
Zurück zum Zitat Salimans T, Kingma DP (2016) In: Advances in neural information processing systems, pp. 901–909 Salimans T, Kingma DP (2016) In: Advances in neural information processing systems, pp. 901–909
20.
Zurück zum Zitat Qing-kun S, Min HAO (2006) Sturctural optimization of BP neural network based on correlation pruning algorithm. Control Theor Appl 25:4–6 Qing-kun S, Min HAO (2006) Sturctural optimization of BP neural network based on correlation pruning algorithm. Control Theor Appl 25:4–6
21.
Zurück zum Zitat Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1, p 3 Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1, p 3
22.
Zurück zum Zitat Ye SJY, Ning G (2016) A research of optimization algorithm in convolution neural network, Qi. Qi Har Univ (Natural science) 32(2):27 Ye SJY, Ning G (2016) A research of optimization algorithm in convolution neural network, Qi. Qi Har Univ (Natural science) 32(2):27
23.
Zurück zum Zitat Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323 Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323
Metadaten
Titel
Fast-Convergent Fully Connected Deep Learning Model Using Constrained Nodes Input
verfasst von
Chen Ding
Ying Li
Lei Zhang
Jinyang Zhang
Lu Yang
Wei Wei
Yong Xia
Yanning Zhang
Publikationsdatum
11.06.2018
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 3/2019
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-018-9872-y

Weitere Artikel der Ausgabe 3/2019

Neural Processing Letters 3/2019 Zur Ausgabe