Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Deep Graph Laplacian Hashing for Image Retrieval Read chapter Read first chapter

2018 | OriginalPaper | Chapter

Sweeper: Design of the Augmented Path in Residual Networks

Authors : Kang Shi, Weiqiang Wang

Published in: Advances in Multimedia Information Processing – PCM 2017

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In recent years, the residual networks (ResNets) have become popular in training very deep neural networks due to its impressive applications in multiple tasks of ILSVRC 2015. In this work, we propose a new network architecture called Sweeper which augments another path in residual block to control the information in residual networks. The proposed architecture is suitable for improving various ResNets. Our experimental results on CIFAR-10 and CIFAR-100 datasets show that the proposed module can bring stable classification accuracy improvement for ResNets and present a new possibility to enhance performance of ResNets differing from deepening and widening it.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Mini Neural Networks for Effective and Efficient Mobile Album Organization
next chapter Sketch Based Model-Like Standing Style Recommendation
Footnotes
1
We refer all shortcuts from the initial layer to global average pooling in ResNets with highway.
 
2
We refer the design of \(3\times 3\) convolution after \(3\times 3\) convolution in residual path with \(\text {B}(3,3)\). Similarly, bottleneck design \( \left[ \begin{aligned} 1\times 1 conv \\ 3\times 3 conv \\ 1\times 1 conv \\ \end{aligned} \right] \) is noted as \(\text {B}(1,3,1)\).
 
3
For a widened ResNet, we denote the network with k-times width in the form of, for example, \(\text {B}(3,3)\)-2 [11].
 
Literature
1.
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
2.
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556 (2014)
3.
4.
Raghu, M., Poole, B., Kleinberg, J., Ganguli, S., Sohl-Dickstein, J.: On the expressive power of deep neural networks. arXiv preprint arXiv:​1606.​05336 (2016)
5.
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
6.
7.
Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
8.
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Computer Science (2015)
9.
Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)CrossRef
10.
Srivastava, R.K., Greff, K., Schmidhuber, J.: Highway networks. In: Computer Science (2015)
11.
12.
Veit, A., Wilber, M.J., Belongie, S.: Residual networks behave like ensembles of relatively shallow networks. In: Advances in Neural Information Processing Systems, pp. 550–558 (2016)
13.
Krizhevsky, A., Hinton, G.: Learning multiple layers of features from tiny images (2009)
14.
15.
16.
Larsson, G., Maire, M., Shakhnarovich, G.: FractalNet: ultra-deep neural networks without residuals. arXiv preprint arXiv:​1605.​07648 (2016)
17.
Glorot, X., Bengio, Y.: Understanding the difficulty of training deep feedforward neural networks. In: AISTATS, vol. 9, pp. 249–256 (2010)
Metadata
Title
Sweeper: Design of the Augmented Path in Residual Networks
Authors
Kang Shi
Weiqiang Wang
Copyright Year
2018
Book
Advances in Multimedia Information Processing – PCM 2017

Print ISBN: 978-3-319-77379-7

Electronic ISBN: 978-3-319-77380-3

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-77380-3_78