Skip to main content
Top
Published in: Neural Processing Letters 6/2022

21-01-2022

EFRNet: Efficient Feature Reuse Network for Real-time Semantic Segmentation

Authors: Yaqian Li, Moran Li, Zhongliang Li, Cunjun Xiao, Haibin Li

Published in: Neural Processing Letters | Issue 6/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Semantic segmentation is a kind of dense prediction task, which has high requirements on the prediction accuracy and inference speed in mobile terminals. To reduce the computational burden of the segmentation network and supplement the missing spatial information of high-level features, an efficient feature reuse network (EFRNet) is proposed in two steps: a Multi-scale Bottleneck module is designed to extract multi-scale features, and a lightweight backbone is designed based on the MB module; then, features of different depths are integrated through efficient feature reuse model. Experiments on Cityscapes datasets demonstrate that the proposed EFRNet achieves an impressive balance between speed and precision. Specifically, without any pre-trained model and post-processing, it achieves 75.58% Mean IoU on the Cityscapes test dataset with the speed of 118 FPS on a single RTX 2080Ti GPU.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans Pattern Anal & Mach Intell 39(2):1–1 Badrinarayanan V, Kendall A, Cipolla R (2017) SegNet: a deep convolutional encoder–decoder architecture for image segmentation. IEEE Trans Pattern Anal & Mach Intell 39(2):1–1
2.
go back to reference Chen LC et al. (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. In: European conference on computer vision Chen LC et al. (2018) Encoder–decoder with atrous separable convolution for semantic image segmentation. In: European conference on computer vision
3.
4.
go back to reference Chen LC et al. (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. Comput Vis Pattern Recogn. arXiv:1412.7062v4 Chen LC et al. (2014) Semantic image segmentation with deep convolutional nets and fully connected CRFs. Comput Vis Pattern Recogn. arXiv:​1412.​7062v4
5.
go back to reference Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR) Chollet F (2017) Xception: deep learning with depthwise separable convolutions. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR)
6.
go back to reference Gao Roland (2021) Rethink dilated convolution for real-time semantic segmentation. In: CoRR abs/2111.09957 Gao Roland (2021) Rethink dilated convolution for real-time semantic segmentation. In: CoRR abs/2111.09957
7.
go back to reference He K. et al. (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR) He K. et al. (2016) Deep residual learning for image recognition. In: 2016 IEEE Conference on computer vision and pattern recognition (CVPR)
8.
go back to reference Huang G et al. (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269 Huang G et al. (2017) Densely connected convolutional networks. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp 2261–2269
9.
go back to reference Jégou S et al. (2016) The one hundred layers tiramisu: fully convolutional denseNets for semantic segmentation. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW) Jégou S et al. (2016) The one hundred layers tiramisu: fully convolutional denseNets for semantic segmentation. In: 2017 IEEE Conference on computer vision and pattern recognition workshops (CVPRW)
10.
go back to reference Jiang W et al. (2020) LRNNet: a light-weighted network with efficient reduced non-local operation for real-time semantic segmentation. In: IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp 1–6 Jiang W et al. (2020) LRNNet: a light-weighted network with efficient reduced non-local operation for real-time semantic segmentation. In: IEEE International Conference on Multimedia & Expo Workshops (ICMEW), pp 1–6
11.
go back to reference Li G et al. (2019) DABNet: Depth-wise asymmetric Bottleneck for real-time semantic segmentation. In: BMVC. BMVA Press, p 259 Li G et al. (2019) DABNet: Depth-wise asymmetric Bottleneck for real-time semantic segmentation. In: BMVC. BMVA Press, p 259
12.
go back to reference Li G. et al. (2020) Depth-wise asymmetric Bottleneck with point-wise aggregation decoder for real-time semantic segmentation in urban scenes. In: IEEE Access vol. 99: pp. 1–1 Li G. et al. (2020) Depth-wise asymmetric Bottleneck with point-wise aggregation decoder for real-time semantic segmentation in urban scenes. In: IEEE Access vol. 99: pp. 1–1
13.
go back to reference Li H. et al. (2020) DFANet: deep feature aggregation for real-time semantic segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR) Li H. et al. (2020) DFANet: deep feature aggregation for real-time semantic segmentation. In: 2019 IEEE/CVF Conference on computer vision and pattern recognition (CVPR)
14.
go back to reference Lin G. et al. (2017) RefineNet: multi-path refinement networks for high-resolution semantic segmentation. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR) Lin G. et al. (2017) RefineNet: multi-path refinement networks for high-resolution semantic segmentation. In: 2017 IEEE Conference on computer vision and pattern recognition (CVPR)
15.
go back to reference Liu M, Yin H (2019) Feature pyramid encoding network for real-time semantic segmentation. BMVC, BMVA Press, p 260 Liu M, Yin H (2019) Feature pyramid encoding network for real-time semantic segmentation. BMVC, BMVA Press, p 260
16.
go back to reference Mehta S. et al. (2018) ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), Springer, Cham Mehta S. et al. (2018) ESPNet: efficient spatial pyramid of dilated convolutions for semantic segmentation. In: Proceedings of the european conference on computer vision (ECCV), Springer, Cham
17.
go back to reference Nirkin Y, Wolf L, Hassner T (2020) HyperSeg: patch-wise hypernetwork for real-time semantic segmentation. In: CoRR abs/2012.11582 Nirkin Y, Wolf L, Hassner T (2020) HyperSeg: patch-wise hypernetwork for real-time semantic segmentation. In: CoRR abs/2012.11582
18.
go back to reference Pan J. et al. (2016) Shallow and deep convolutional networks for saliency prediction. In: Computer vision & pattern recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition Pan J. et al. (2016) Shallow and deep convolutional networks for saliency prediction. In: Computer vision & pattern recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition
19.
go back to reference Paszke A et al. (2016) ENet: a deep neural network architecture for real-time semantic segmentation. Comput Vis Pattern Recogn. arXiv:1606.02147 Paszke A et al. (2016) ENet: a deep neural network architecture for real-time semantic segmentation. Comput Vis Pattern Recogn. arXiv:​1606.​02147
20.
go back to reference Poudel RP et al. (2018) ContextNet: exploring context and detail for semantic segmentation in real-time. In: BMVC, BMVA Press, p 146 Poudel RP et al. (2018) ContextNet: exploring context and detail for semantic segmentation in real-time. In: BMVC, BMVA Press, p 146
21.
go back to reference Romera E et al (2017) ERFNet: efficient residual factorized convNet for real-time semantic segmentation. IEEE Trans Intell Transport Syst 19(1):263–272CrossRef Romera E et al (2017) ERFNet: efficient residual factorized convNet for real-time semantic segmentation. IEEE Trans Intell Transport Syst 19(1):263–272CrossRef
22.
go back to reference Sandler M. et al. (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition (CVPR) Sandler M. et al. (2018) MobileNetV2: inverted residuals and linear bottlenecks. In: 2018 IEEE/CVF Conference on computer vision and pattern recognition (CVPR)
23.
go back to reference Shrivastava A, Gupta A, Girshick RB (2016) Training region-based object detectors with online hard example mining. In: IEEE conference on computer vision and pattern recognition pp. 761–769 Shrivastava A, Gupta A, Girshick RB (2016) Training region-based object detectors with online hard example mining. In: IEEE conference on computer vision and pattern recognition pp. 761–769
24.
go back to reference Si H et al. (2019) Real-time semantic segmentation via multiply spatial fusion network. BMVC, BMVA Press Si H et al. (2019) Real-time semantic segmentation via multiply spatial fusion network. BMVC, BMVA Press
25.
go back to reference Siam M. et al. (2018) RTSeg: real-time semantic segmentation comparative study. In: Under Review by ICIP 2018 Siam M. et al. (2018) RTSeg: real-time semantic segmentation comparative study. In: Under Review by ICIP 2018
26.
go back to reference Wang P. et al. (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE Winter conference on applications of computer vision (WACV) Wang P. et al. (2018) Understanding convolution for semantic segmentation. In: 2018 IEEE Winter conference on applications of computer vision (WACV)
27.
go back to reference Wang Y. et al. (2019) LEDNet: a lightweight encoder–decoder network for real-time semantic segmentation. In: IEEE International conference on image processing (ICIP) Wang Y. et al. (2019) LEDNet: a lightweight encoder–decoder network for real-time semantic segmentation. In: IEEE International conference on image processing (ICIP)
28.
go back to reference Yang M. et al. (2018) DenseASPP for Semantic Segmentation in Street Scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition Yang M. et al. (2018) DenseASPP for Semantic Segmentation in Street Scenes. In: Proceedings of the IEEE conference on computer vision and pattern recognition
29.
go back to reference Yu C et al. (2021) BiSeNet V2: bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vis 129:3051–3068 Yu C et al. (2021) BiSeNet V2: bilateral network with guided aggregation for real-time semantic segmentation. Int J Comput Vis 129:3051–3068
30.
go back to reference Yu C. et al. (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: European conference on computer vision Yu C. et al. (2018) BiSeNet: bilateral segmentation network for real-time semantic segmentation. In: European conference on computer vision
31.
go back to reference Yu J. et al. (2019) Hierarchical deep click feature prediction for fine-grained image recognition. In: IEEE Trans Pattern Anal Mach Intell PP.99, pp. 1–1 Yu J. et al. (2019) Hierarchical deep click feature prediction for fine-grained image recognition. In: IEEE Trans Pattern Anal Mach Intell PP.99, pp. 1–1
32.
go back to reference Zhang J, Cao Y, Wu Q (2021) Vector of locally and adaptively aggregated descriptors for image feature representation. Pattern Recog 116:107952CrossRef Zhang J, Cao Y, Wu Q (2021) Vector of locally and adaptively aggregated descriptors for image feature representation. Pattern Recog 116:107952CrossRef
33.
go back to reference Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432MathSciNetCrossRefMATH Zhang J, Yu J, Tao D (2018) Local deep-feature alignment for unsupervised dimension reduction. IEEE Trans Image Process 27(5):2420–2432MathSciNetCrossRefMATH
34.
go back to reference Zhang Z et al. (2018) ExFuse: enhancing feature fusion for semantic segmentation. In: European conference on computer vision Zhang Z et al. (2018) ExFuse: enhancing feature fusion for semantic segmentation. In: European conference on computer vision
35.
go back to reference Zhao H et al. (2018) ICNet for real-time semantic segmentation on high-resolution images. Lecture Notes in Computer Science, vol 11207. Springer, pp 418–434 Zhao H et al. (2018) ICNet for real-time semantic segmentation on high-resolution images. Lecture Notes in Computer Science, vol 11207. Springer, pp 418–434
Metadata
Title
EFRNet: Efficient Feature Reuse Network for Real-time Semantic Segmentation
Authors
Yaqian Li
Moran Li
Zhongliang Li
Cunjun Xiao
Haibin Li
Publication date
21-01-2022
Publisher
Springer US
Published in
Neural Processing Letters / Issue 6/2022
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10740-w

Other articles of this Issue 6/2022

Neural Processing Letters 6/2022 Go to the issue