Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 3/2021

05.09.2020 | Original Article

Multipath feature recalibration DenseNet for image classification

verfasst von: Bolin Chen, Tiesong Zhao, Jiahui Liu, Liqun Lin

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 3/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Recently, deep neural networks have demonstrated their efficiency in image classification tasks, which are commonly achieved by an extended depth and width of network architecture. However, poor convergence, over-fitting and gradient disappearance might be generated with such comprehensive architectures. Therefore, DenseNet is developed to address these problems. Although DenseNet adopts bottleneck technique in DenseBlocks to avoid relearning feature-maps and decrease parameters, this operation may lead to the skip and loss of important features. Besides, it still takes oversized computational power when the depth and width of the network architecture are increased for better classification. In this paper, we propose a variate of DenseNet, named Multipath Feature Recalibration DenseNet (MFR-DenseNet), to stack convolution layers instead of adopting bottleneck for improving feature extraction. Meanwhile, we build multipath DenseBlocks with Squeeze-Excitation (SE) module to represent the interdependencies of useful feature-maps among different DenseBlocks. Experiments in CIFAR-10, CIFAR-100, MNIST and SVHN reveal the efficiency of our network, with further reduced redundancy whilst maintaining the high accuracy of DenseNet.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Neural information processing systems. Springer, New York, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Neural information processing systems. Springer, New York, pp 1097–1105
2.
Zurück zum Zitat Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR) Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations (ICLR)
3.
Zurück zum Zitat Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition(CVPR), pp 1–9 Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition(CVPR), pp 1–9
4.
Zurück zum Zitat Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. Advances in neural information processing systems. Springer, New York, pp 2377–2385 Srivastava RK, Greff K, Schmidhuber J (2015) Training very deep networks. Advances in neural information processing systems. Springer, New York, pp 2377–2385
5.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 770–778
6.
Zurück zum Zitat Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertain Fuzziness Knowl Based Syst 06(02):107–116CrossRef Hochreiter S (1998) The vanishing gradient problem during learning recurrent neural nets and problem solutions. Int J Uncertain Fuzziness Knowl Based Syst 06(02):107–116CrossRef
7.
Zurück zum Zitat Bengio Y (2002) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw Learn Syst 05(02):157–166CrossRef Bengio Y (2002) Learning long-term dependencies with gradient descent is difficult. IEEE Trans Neural Netw Learn Syst 05(02):157–166CrossRef
8.
Zurück zum Zitat Huang G, Liu Z, Laurens VDM, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4700–4708 Huang G, Liu Z, Laurens VDM, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 4700–4708
9.
Zurück zum Zitat Liu W, Zeng K (2018) Sparsenet: a sparse DenseNet for image classification, arXiv. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR) Liu W, Zeng K (2018) Sparsenet: a sparse DenseNet for image classification, arXiv. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR)
10.
Zurück zum Zitat Zhang K, Guo Y, Wang X, Yuan J, Ding Q (2019) Multiple feature reweight DenseNet for image classification. IEEE Access 7:9872–9880CrossRef Zhang K, Guo Y, Wang X, Yuan J, Ding Q (2019) Multiple feature reweight DenseNet for image classification. IEEE Access 7:9872–9880CrossRef
11.
Zurück zum Zitat Zhang Z, Liang X, Dong X, Xie Y, Cao G (2018) A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution. IEEE Trans Med Imaging 37(6):1407–1417CrossRef Zhang Z, Liang X, Dong X, Xie Y, Cao G (2018) A sparse-view CT reconstruction method based on combination of DenseNet and deconvolution. IEEE Trans Med Imaging 37(6):1407–1417CrossRef
12.
Zurück zum Zitat Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Neural information processing systems (NeurIPS), pp 4467–4475 Chen Y, Li J, Xiao H, Jin X, Yan S, Feng J (2017) Dual path networks. In: Neural information processing systems (NeurIPS), pp 4467–4475
13.
Zurück zum Zitat Lodhi B, Kang J (2019) Multipath-densenet: a supervised ensemble architecture of densely connected convolutional networks. Inf Sci 482:63–72CrossRef Lodhi B, Kang J (2019) Multipath-densenet: a supervised ensemble architecture of densely connected convolutional networks. Inf Sci 482:63–72CrossRef
14.
Zurück zum Zitat Huang G, Liu S, Der Maaten LV, Weinberger KQ (2018) CondenseNet: an efficient DenseNet using learned group convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2752–2761 Huang G, Liu S, Der Maaten LV, Weinberger KQ (2018) CondenseNet: an efficient DenseNet using learned group convolutions. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 2752–2761
15.
Zurück zum Zitat Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning (ICML), pp 448–456 Ioffe S, Szegedy C (2015) Batch normalization: accelerating deep network training by reducing internal covariate shift. In: International conference on machine learning (ICML), pp 448–456
16.
Zurück zum Zitat Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: International conference on artificial intelligence and statistics, pp 315–323 Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: International conference on artificial intelligence and statistics, pp 315–323
17.
Zurück zum Zitat Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef Lecun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324CrossRef
18.
Zurück zum Zitat Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7132–7141 Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 7132–7141
19.
Zurück zum Zitat Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images, Tech Report Krizhevsky A, Hinton G (2009) Learning multiple layers of features from tiny images, Tech Report
20.
Zurück zum Zitat Deng L (2012) The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process Mag 29(6):141–142CrossRef Deng L (2012) The MNIST database of handwritten digit images for machine learning research [best of the web]. IEEE Signal Process Mag 29(6):141–142CrossRef
22.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: The IEEE international conference on computer vision (ICCV), pp 1026–1034 He K, Zhang X, Ren S, Sun J (2015) Delving deep into rectifiers: surpassing human-level performance on imagenet classification. In: The IEEE international conference on computer vision (ICCV), pp 1026–1034
23.
Zurück zum Zitat Lin M, Chen Q, Yan S (2014) Network in network. In: International conference on learning representations (ICLR) Lin M, Chen Q, Yan S (2014) Network in network. In: International conference on learning representations (ICLR)
24.
Zurück zum Zitat Larsson G, Maire M, Shakhnarovich G (2017) FractalNet: ultra-deep neural networks without residuals. In: International conference on learning representations (ICLR) Larsson G, Maire M, Shakhnarovich G (2017) FractalNet: ultra-deep neural networks without residuals. In: International conference on learning representations (ICLR)
25.
Zurück zum Zitat Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 5987–5995 Xie S, Girshick R, Dollar P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 5987–5995
26.
Zurück zum Zitat Zagoruyko S, Komodakis N (2016) Wide residual networks. In: Proceedings of the British Machine Vision Conference (BMVC), pp 87.1–87.12 Zagoruyko S, Komodakis N (2016) Wide residual networks. In: Proceedings of the British Machine Vision Conference (BMVC), pp 87.1–87.12
27.
Zurück zum Zitat Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2019) Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images. Comput Biol Med 107:45–47 Sandler M, Howard A, Zhu M, Zhmoginov A, Chen LC (2019) Grading of hepatocellular carcinoma using 3D SE-DenseNet in dynamic enhanced MR images. Comput Biol Med 107:45–47
Metadaten
Titel
Multipath feature recalibration DenseNet for image classification
verfasst von
Bolin Chen
Tiesong Zhao
Jiahui Liu
Liqun Lin
Publikationsdatum
05.09.2020
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 3/2021
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-020-01194-4

Weitere Artikel der Ausgabe 3/2021

International Journal of Machine Learning and Cybernetics 3/2021 Zur Ausgabe

Neuer Inhalt