Skip to main content
Erschienen in: Cluster Computing 2/2019

09.12.2017

Pyramidal RoR for image classification

verfasst von: Ke Zhang, Liru Guo, Ce Gao, Zhenbing Zhao

Erschienen in: Cluster Computing | Sonderheft 2/2019

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The Residual Networks of Residual Networks (RoR) exhibits excellent performance in the image classification task, but sharply increasing the number of feature map channels makes the characteristic information transmission incoherent, which losses a certain of information related to classification prediction, limiting the classification performance. In this paper, a Pyramidal RoR network model is proposed by analysing the characteristics of RoR and combining with the PyramidNet. Firstly, based on RoR, the Pyramidal RoR network model with channels gradually increasing is designed. Secondly, we analysed the effect of different residual block structures on performance, and chosen the residual block structure which best favoured the classification performance. Finally, we add an important principle to further optimize Pyramidal RoR networks, drop-path is used to avoid over-fitting and save training time. In this paper, image classification experiments were performed on CIFAR-10/100, SVHN and Adience datasets, and we achieved the current lowest classification error rates were 2.96, 16.40 and 1.59% on CIFAR-10/100 and SVHN, respectively. Experiments show that the Pyramidal RoR network optimization method can improve the network performance for image classification and effectively suppress the gradient disappearance problem in DCNN training.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015) LeCun, Y., Bengio, Y., Hinton, G.: Deep learning. Nature 521(7553), 436–444 (2015)
2.
Zurück zum Zitat Zou, W.Y., Wang, X.Y., Sun, M., Lin, Y.: Generic object detection with dense neural patterns and regional. arXiv preprint arXiv:1404.4316 (2014) Zou, W.Y., Wang, X.Y., Sun, M., Lin, Y.: Generic object detection with dense neural patterns and regional. arXiv preprint arXiv:​1404.​4316 (2014)
3.
Zurück zum Zitat Krizhenvshky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional networks. In: Proceedings of the Advances in Neural Information Processing System, pp. 1097–1105 (2012) Krizhenvshky, A., Sutskever, I., Hinton, G.: Imagenet classification with deep convolutional networks. In: Proceedings of the Advances in Neural Information Processing System, pp. 1097–1105 (2012)
4.
Zurück zum Zitat Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A., Fei-Fei, L.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2014) Russakovsky, O., Deng, J., Su, H., Krause, J., Satheesh, S., Ma, S., Huang, Z., Karpathy, A., Khosla, A., Bernstein, M., Berg, A., Fei-Fei, L.: Imagenet large scale visual recognition challenge. Int. J. Comput. Vis. 115, 211–252 (2014)
5.
Zurück zum Zitat Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition, arXiv preprint arXiv:​1409.​1556 (2014)
6.
Zurück zum Zitat Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recogniton, pp. 1–9 (2015) Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recogniton, pp. 1–9 (2015)
7.
8.
Zurück zum Zitat He, K., Sun, J.: Convolutional neural networks at constrained time cost, In: Proceedings of the IEEE Conference on Computer and Vision Pattern Recognition, pp. 5353–5360 (2015) He, K., Sun, J.: Convolutional neural networks at constrained time cost, In: Proceedings of the IEEE Conference on Computer and Vision Pattern Recognition, pp. 5353–5360 (2015)
10.
12.
Zurück zum Zitat Zhang, K., Sun, M., Han, X., et al.: Residual networks of residual networks: multilevel residual networks. IEEE Trans. Circuit Syst. Video Technol. 99, 1 (2016) Zhang, K., Sun, M., Han, X., et al.: Residual networks of residual networks: multilevel residual networks. IEEE Trans. Circuit Syst. Video Technol. 99, 1 (2016)
13.
Zurück zum Zitat Clevert, D.-A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus), arXiv preprint arXiv:1511.07289 (2015) Clevert, D.-A., Unterthiner, T., Hochreiter, S.: Fast and accurate deep network learning by exponential linear units (elus), arXiv preprint arXiv:​1511.​07289 (2015)
14.
Zurück zum Zitat Trottier, L., Giguere, P., Chaib-draa, B.: Parametric exponential linear unit for deep convolutional neural networks, arXiv preprint arXiv:1605.09322 (2016) Trottier, L., Giguere, P., Chaib-draa, B.: Parametric exponential linear unit for deep convolutional neural networks, arXiv preprint arXiv:​1605.​09322 (2016)
16.
Zurück zum Zitat Krizhenvshky, A., Hinton, G.: Learning multiple layers of features from tiny images, M.Sc. thesis, Deptartment of Computer Science, University of Toronto, Toronto, ON, Canada (2009) Krizhenvshky, A., Hinton, G.: Learning multiple layers of features from tiny images, M.Sc. thesis, Deptartment of Computer Science, University of Toronto, Toronto, ON, Canada (2009)
17.
Zurück zum Zitat Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning, In: Proceedings of the NIPS Workshop Deep Learning and Unsupervised Feature Learning, pp. 1–9 (2011) Netzer, Y., Wang, T., Coates, A., Bissacco, A., Wu, B., Ng, A.Y.: Reading digits in natural images with unsupervised feature learning, In: Proceedings of the NIPS Workshop Deep Learning and Unsupervised Feature Learning, pp. 1–9 (2011)
18.
Zurück zum Zitat Eidinger, E., Enbar, R., Hassner, T.: Age and gender estimation of unfiltered faces[J]. IEEE Trans. Inf. Forensics Secur. 9(12), 2170–2179 (2014) Eidinger, E., Enbar, R., Hassner, T.: Age and gender estimation of unfiltered faces[J]. IEEE Trans. Inf. Forensics Secur. 9(12), 2170–2179 (2014)
19.
Zurück zum Zitat Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (2014) Bengio, Y., Simard, P., Frasconi, P.: Learning long-term dependencies with gradient descent is difficult. IEEE Trans. Neural Netw. 5(2), 157–166 (2014)
21.
Zurück zum Zitat Nair, V., Hinton, G.: Rectified linear units improve restricted Boltzmann machines, In: Proceedings of the International Conference on ICML pp. 807–814 (2010) Nair, V., Hinton, G.: Rectified linear units improve restricted Boltzmann machines, In: Proceedings of the International Conference on ICML pp. 807–814 (2010)
22.
Zurück zum Zitat Han, D., Kim, J., Kim, J.: Deep pyramidal residual networks, In: Proceedings of the International Conference on CVPR (2017) Han, D., Kim, J., Kim, J.: Deep pyramidal residual networks, In: Proceedings of the International Conference on CVPR (2017)
23.
24.
Zurück zum Zitat Wang, F., Jiang, M., Qian C, et al.: Residual Attention Network for Image Classification, In: Proceedings of the International Conference on CVPR (2017) Wang, F., Jiang, M., Qian C, et al.: Residual Attention Network for Image Classification, In: Proceedings of the International Conference on CVPR (2017)
25.
Zurück zum Zitat Chen, Y., Li, J., Xiao, H, et al.: Dual Path Networks, In: Proceedings of the International Conference on CVPR (2017) Chen, Y., Li, J., Xiao, H, et al.: Dual Path Networks, In: Proceedings of the International Conference on CVPR (2017)
26.
Zurück zum Zitat Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:1502.03167 (2015) Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:​1502.​03167 (2015)
27.
Zurück zum Zitat Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:1502.03167 (2015) Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift, arXiv preprint arXiv:​1502.​03167 (2015)
28.
Zurück zum Zitat Hinton, G., Srivastava, N., Krizhevsky, A., Weinberger, K.: Improving neural networks by preventing co-adaptation of feature detectors, arXiv preprint arXiv:1207.0580 (2012) Hinton, G., Srivastava, N., Krizhevsky, A., Weinberger, K.: Improving neural networks by preventing co-adaptation of feature detectors, arXiv preprint arXiv:​1207.​0580 (2012)
29.
Zurück zum Zitat Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014) Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15, 1929–1958 (2014)
32.
Zurück zum Zitat Larsson, G., Maire, M., Shakhnarovich, G.: FractalNet: ultra-deep neural networks without residuals, arXiv preprint arXiv:1605.07648 (2016) Larsson, G., Maire, M., Shakhnarovich, G.: FractalNet: ultra-deep neural networks without residuals, arXiv preprint arXiv:​1605.​07648 (2016)
33.
Zurück zum Zitat Shah, A., Shinde, S., Kadam, E., Shah, H.: Deep residual networks with exponential linear unit, arXiv preprint arXiv:1604.04112 (2016) Shah, A., Shinde, S., Kadam, E., Shah, H.: Deep residual networks with exponential linear unit, arXiv preprint arXiv:​1604.​04112 (2016)
34.
36.
Zurück zum Zitat Xie, S., Girshick, R., Dollr P., et al.: Aggregated residual transformations for deep neural networks, arXiv preprint arXiv:1611.05431 (2016) Xie, S., Girshick, R., Dollr P., et al.: Aggregated residual transformations for deep neural networks, arXiv preprint arXiv:​1611.​05431 (2016)
37.
Zurück zum Zitat Yamada, Y., Iwamura, M., Kise, K.: Deep pyramidal residual networks with separated stochastic depth[J]. arXiv preprint arXiv:1612.01230 (2016) Yamada, Y., Iwamura, M., Kise, K.: Deep pyramidal residual networks with separated stochastic depth[J]. arXiv preprint arXiv:​1612.​01230 (2016)
Metadaten
Titel
Pyramidal RoR for image classification
verfasst von
Ke Zhang
Liru Guo
Ce Gao
Zhenbing Zhao
Publikationsdatum
09.12.2017
Verlag
Springer US
Erschienen in
Cluster Computing / Ausgabe Sonderheft 2/2019
Print ISSN: 1386-7857
Elektronische ISSN: 1573-7543
DOI
https://doi.org/10.1007/s10586-017-1443-x

Weitere Artikel der Sonderheft 2/2019

Cluster Computing 2/2019 Zur Ausgabe