Top

Neural Processing Letters

Published in:

11-06-2018

Fast-Convergent Fully Connected Deep Learning Model Using Constrained Nodes Input

Authors: Chen Ding, Ying Li, Lei Zhang, Jinyang Zhang, Lu Yang, Wei Wei, Yong Xia, Yanning Zhang

Published in: Neural Processing Letters | Issue 3/2019

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Recently, deep learning models exhibit promising performance in various applications. However, most of them converge slowly due to gradient vanishing. To address this problem, we propose a fast convergent fully connected deep learning network in this study. Through constraining the input values of nodes on the fully connected layers, the proposed method is able to well mitigate the gradient vanishing problems in training phase, and thus greatly reduces the training iterations required to reach convergence. Nevertheless, the drop of generalization performance is negligible. Experimental results validate the effectiveness of the proposed method.

previous article Binary Filter for Fast Vessel Pattern Extraction

next article Obstacle Detection by Fusing Point Clouds and Monocular Image

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504MathSciNetCrossRefMATH

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105

Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2014) Going Deeper with Convolutions, arXiv preprint arXiv:1409.4842

He K, Zhang X, Ren S, Sun J (2016) In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778

Yang W, Zhang H, Yang J, Wu J, Yin X, Chen Y, Shu H, Luo L, Coatrieux G, Gui Z (2017) Improving low-dose ct image using residual convolutional network. IEEE Access 5:24698CrossRef

Hu J, Shen L, Sun G (2017) Squeeze-and-excitation networks, arXiv preprint arXiv:1709.01507

Chan TH, Jia K, Gao S, Lu J, Zeng Z, Ma Y (2014) Pcanet: a simple deep learning baseline for image classification? IEEE Trans Image Process 24(12):5017MathSciNetCrossRefMATH

Zeng R, Wu J, Shao Z, Chen Y, Chen B, Senhadji L, Shu H (2016) Color image classification via quaternion principal component analysis network. Neurocomputing 216:416CrossRef

10.

Ding C, Li Y, Xia Y, Wei W, Zhang L, Zhang Y (2017) Convolutional neural networks based hyperspectral image classification method with adaptive kernels. Remote Sens 9(6):618CrossRef

11.

Zhang L, Wei W, Zhang Y, Shen C, Hengel AVD, Shi Q (2018) Cluster sparsity field: an internal hyperspectral imagery prior for reconstruction. Int J Comput Vis. https://doi.org/10.1007/s11263-018-1080-8

12.

Wei W, Zhang L, Tian C, Plaza A, Zhang Y (2017) Structured sparse coding-based hyperspectral imagery denoising with intracluster filtering. IEEE Trans on Geosci Remote Sens 55(12):6860CrossRef

13.

Wang C, Zhang L, Wei W, Zhang Y (2018) When low rank representation based hyperspectral imagery classification meets segmented stacked denoising auto-encoder based spatial-spectral feature. Remote Sens 10(2):284CrossRef

14.

Krger J, Westermann R (2003) In: ACM SIGGRAPH, pp 908–916

15.

Byong-Heon K, Burm-Suk S (2005) Design and implementation of jpeg image display board using FFGA. J Digit Contents Soc 6(3):169–174

16.

Le QV, Ngiam J, Coates A, Lahiri A, Prochnow B, Ng AY (2011) In: International conference on machine learning, ICML 2011, Bellevue, Washington, Usa, June 28–July, pp. 265–272

17.

Orr GB, Müller KR (1998) Neural networks: tricks of the trade. Can J Anaesth 41(7):658

18.

Salimans T, Kingma DP (2016) In: Advances in neural information processing systems, pp. 901–909

19.

Ba JL, Kiros JR, Hinton GE (2016) Layer normalization, arXiv preprint arXiv:1607.06450

20.

Qing-kun S, Min HAO (2006) Sturctural optimization of BP neural network based on correlation pruning algorithm. Control Theor Appl 25:4–6

21.

Huang G, Liu Z, Weinberger KQ, van der Maaten L (2017) In: Proceedings of the IEEE conference on computer vision and pattern recognition, vol 1, p 3

22.

Ye SJY, Ning G (2016) A research of optimization algorithm in convolution neural network, Qi. Qi Har Univ (Natural science) 32(2):27

23.

Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp 315–323

Title: Fast-Convergent Fully Connected Deep Learning Model Using Constrained Nodes Input
Authors: Chen Ding
Ying Li
Lei Zhang
Jinyang Zhang
Lu Yang
Wei Wei
Yong Xia
Yanning Zhang
Publication date: 11-06-2018
Publisher: Springer US
Published in: Neural Processing Letters / Issue 3/2019
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI: https://doi.org/10.1007/s11063-018-9872-y

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 3/2019

Multi-Delay-Dependent Exponential Synchronization for Neutral-Type Stochastic Complex Networks with Markovian Jump Parameters via Adaptive Control

Impact Analysis of the Memristor Failure on Real-Time Control System of Robotic Arm

Weighted Pseudo Almost Periodic Solutions for Cellular Neural Networks with Multi-proportional Delays

Multi-task Character-Level Attentional Networks for Medical Concept Normalization

A Dynamic ELM with Balanced Variance and Bias for Long-Term Online Prediction

Pseudo Almost Automorphic Solutions for Multidirectional Associative Memory Neural Network with Mixed Delays