Skip to main content
SpringerLink
Log in

Robust detection method for improving small traffic sign recognition based on spatial pyramid pooling

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

An extraordinary challenge for real-world applications is traffic sign recognition, which plays a crucial role in driver guidance. Traffic signals are very difficult to detect using an extremely precise, real-time approach in practical autonomous driving scenes. This article reviews several object detection methods, including Yolo V3 and Densenet, in conjunction with spatial pyramid pooling (SPP). The SPP principle is employed to boost the Yolo V3 and Densenet backbone networks to extract the features. Moreover, we adopt spatial pyramid pooling to learn object features more completely. These models are measured and compared with key measurement parameters such as average accuracy (mAP), working area size, detection time, and billion floating-point number (BFLOPS). Based on the experimental results, Yolo V3 SPP outperforms state-of-the-art systems. Specifically, Yolo V3 SPP obtains 87.8% accuracy for small (S) target, 98.0% for medium (M) target, and 98.6% for large target groups in the BTSD dataset. Our results have shown that Yolo V3 SPP obtains the highest total BFLOPS (66.111), and mAP (99.28%). Consequently, SPP upgrades the achievement of all experimental models.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  • Arcos-García Á, Álvarez-García JA, Soria-Morillo LM (2018) Evaluation of deep neural networks for traffic sign detection systems. Neurocomputing 316:332–344. https://doi.org/10.1016/j.neucom.2018.08.009

    Article  Google Scholar 

  • Bangquan X, Xiong WX (2019) Real-time embedded traffic sign recognition using efficient convolutional neural network. IEEE Access 7:53330–53346. https://doi.org/10.1109/ACCESS.2019.2912311

    Article  Google Scholar 

  • Basbug AM, Sert M (2019) Acoustic scene classification using spatial pyramid pooling with convolutional neural networks. In: Proceedings—13th IEEE international conference on semantic computing, ICSC 2019, pp 128–131

  • Chen T, Lu S (2016) Accurate and efficient traffic sign detection using discriminative AdaBoost and support vector regression. IEEE Trans Veh Technol 65:4006–4015. https://doi.org/10.1109/TVT.2015.2500275

    Article  Google Scholar 

  • Chen CH, Chen CY, Liu NY (2019a) Hardware design of codebook-based moving object detecting method for dynamic gesture recognition. Intell Autom Soft Comput. https://doi.org/10.31209/2019.100000099

    Article  Google Scholar 

  • Chen H, He Z, Shi B, Zhong T (2019b) Research on recognition method of electrical components based on YOLO V3. IEEE Access 7:157818–157829. https://doi.org/10.1109/ACCESS.2019.2950053

    Article  Google Scholar 

  • Chen RC, Dewi C, Huang SW, Caraka RE (2020) Selecting critical features for data classification based on machine learning methods. J Big Data 7:1–26. https://doi.org/10.1186/s40537-020-00327-4

    Article  Google Scholar 

  • Dewi C, Chen R-C (2019) Random forest and support vector machine on features selection for regression analysis. Int J Innov Comput Inf Control 15:2027–2038

    Google Scholar 

  • Dewi C, Chen R-C, Hendry, Liu Y-T (2019) Similar music instrument detection via deep convolution YOLO-generative adversarial network. In: 2019 IEEE 10th international conference on awareness science and technology (iCAST), pp 1–6. https://doi.org/10.1109/ICAwST.2019.8923404

  • Dewi C, Chen R-C, Tai S-K (2020a) Evaluation of robust spatial pyramid pooling based on convolutional neural network for traffic sign recognition system. Electronics 9:889. https://doi.org/10.3390/electronics9060889

    Article  Google Scholar 

  • Dewi C, Chen RC, Liu Y-T (2020b) Taiwan stop sign recognition with customize anchor. In: ICCMS ’20, February 26–28, 2020, Brisbane, QLD, Australia, pp 51–55

  • Dewi C, Chen RC, Yu H (2020c) Weight analysis for various prohibitory sign detection and recognition using deep learning. Multim Tools Appl 79:32897–32915. https://doi.org/10.1007/s11042-020-09509-x

    Article  Google Scholar 

  • Dewi C, Chen R-C, Liu Y-T et al (2021a) Yolo V4 for advanced traffic sign recognition with synthetic training data generated by various GAN. IEEE Access. https://doi.org/10.1109/ACCESS.2021.3094201

    Article  Google Scholar 

  • Dewi C, Chen R-C, Liu Y-T, Tai S-K (2021b) Synthetic data generation using DCGAN for improved traffic sign recognition. Neural Comput Appl 33:1–15

    Google Scholar 

  • Dewi C, Chen R, Liu Y, Yu H (2021c) Various generative adversarial networks model for synthetic prohibitory sign image generation. Appl Sci 11:2913

    Article  Google Scholar 

  • Fang W, Wang L, Ren P (2020) Tinier-YOLO: a real-time object detection method for constrained environments. IEEE Access 8:1935–1944. https://doi.org/10.1109/ACCESS.2019.2961959

    Article  Google Scholar 

  • Fatmehsari YR, Ghahari A, Zoroofi RA (2010) Gabor wavelet for road sign detection and recognition using a hybrid classifier. In: MCIT’2010: International Conference on Multimedia Computing and Information Technology. pp 25–28

  • Feng R, Fan C, Li Z, Chen X (2020) Mixed Road User Trajectory Extraction From Moving Aerial Videos Based on Convolution Neural Network Detection. IEEE Access 8:43508–43519. https://doi.org/10.1109/access.2020.2976890

    Article  Google Scholar 

  • Fu Z, Yang Y, Shu C et al (2015) Improved single image dehazing using dark channel prior. J Syst Eng Electron 26:1070–1079. https://doi.org/10.1109/JSEE.2015.00116

    Article  Google Scholar 

  • Fu X, Liang B, Huang Y et al (2020) Lightweight Pyramid Networks for Image Deraining. IEEE Transactions on Neural Networks and Learning Systems 31:1794–1807. https://doi.org/10.1109/TNNLS.2019.2926481

    Article  Google Scholar 

  • Ghatwary N, Ye X, Zolgharni M (2019) Esophageal Abnormality Detection Using DenseNet Based Faster R-CNN with Gabor Features. IEEE Access 7:84374–84385. https://doi.org/10.1109/ACCESS.2019.2925585

    Article  Google Scholar 

  • Grauman K, Darrell T (2005) The pyramid match kernel: Discriminative classification with sets of image features. In: Proceedings of the IEEE International Conference on Computer Vision. pp 1458–1465

  • He K, Zhang X, Ren S, Sun J (2015) Spatial pyramid pooling in deep convolutional networks for visual recognition. IEEE Trans Pattern Anal Mach Intell. https://doi.org/10.1109/TPAMI.2015.2389824

    Article  Google Scholar 

  • Huang Y-Q, Zheng J-C, Sun S-D et al (2020a) Optimized YOLOv3 Algorithm and Its Application in Traffic Flow Detections. Appl Sci 10:1–15. https://doi.org/10.3390/app10093079

    Article  Google Scholar 

  • Huang Z, Zhu X, Ding M, Zhang X (2020b) Medical Image Classification Using a Light-Weighted Hybrid Neural Network Based on PCANet and DenseNet. IEEE Access 8:24697–24712. https://doi.org/10.1109/ACCESS.2020.2971225

    Article  Google Scholar 

  • Huang L, Pun CM (2019) Audio Replay Spoof Attack Detection Using Segment-based Hybrid Feature and DenseNet-LSTM Network. In: ICASSP, IEEE International Conference on Acoustics, Speech and Signal Processing - Proceedings. pp 2567–2571

  • Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017. pp 2261–2269

  • Kabir MH, Thapa K, Yang JY, Yang SH (2019) State-space based linear modeling for human activity recognition in smart space. Intell Autom Soft Comput. https://doi.org/10.31209/2018.100000035

    Article  Google Scholar 

  • Kang H, Chen C (2020) Fast implementation of real-time fruit detection in apple orchards using deep learning. Comput Electron Agric 168:1–10. https://doi.org/10.1016/j.compag.2019.105108

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017a) ImageNet classification with deep convolutional neural networks. Commun ACM 60:84–90. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  • Krizhevsky A, Sutskever I, Hinton GE (2017b) ImageNet classification with deep convolutional neural networks. Commun ACM 120:1–9. https://doi.org/10.1145/3065386

    Article  Google Scholar 

  • Lazebnik S, Schmid C, Ponce J (2006) Beyond bags of features: spatial pyramid matching for recognizing natural scene categories. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 1–8

  • Li J, Li H, Cui G et al (2020) GACNet: a generative adversarial capsule network for regional epitaxial traffic flow prediction. Comput Mater Contin. https://doi.org/10.32604/CMC.2020.09903

    Article  Google Scholar 

  • Liu J, Huang Y, Peng J et al (2018) Fast object detection at constrained energy. IEEE Trans Emerg Top Comput 6:409–416. https://doi.org/10.1109/TETC.2016.2577538

    Article  Google Scholar 

  • Liu C, Li S, Chang F, Wang Y (2019) Machine vision based traffic sign detection methods: review, analyses and perspectives. IEEE Access 7:86578–86596. https://doi.org/10.1109/access.2019.2924947

    Article  Google Scholar 

  • Liu W, Anguelov D, Erhan D, et al (2016) SSD: Single shot multibox detector. In: Lecture notes in computer science (including subseries lecture notes in artificial intelligence and lecture notes in bioinformatics), pp 21–37

  • Mao QC, Sun HM, Liu YB, Jia RS (2019) Mini-YOLOv3: real-time object detector for embedded applications. IEEE Access 7:133529–133538. https://doi.org/10.1109/ACCESS.2019.2941547

    Article  Google Scholar 

  • Mathias M, Timofte R, Benenson R, Van Gool L (2013) Traffic sign recognition—how far are we from the solution? In: Proceedings of the international joint conference on neural networks, pp 1–8

  • Ou Z, Xiao F, Xiong B et al (2019) FAMN: feature aggregation multipath network for small traffic sign detection. IEEE Access 7:178798–178810. https://doi.org/10.1109/ACCESS.2019.2959015

    Article  Google Scholar 

  • Penna GD, Orefice S (2019) Using spatial relations for qualitative specification of gestures. Comput Syst Sci Eng. https://doi.org/10.32604/csse.2019.34.325

    Article  Google Scholar 

  • Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In: Proceedings—30th IEEE conference on computer vision and pattern recognition, CVPR 2017, pp 6517–6525

  • Redmon J, Farhadi A (2018) YOLOv3: an incremental improvement. CoRR abs/1804.0:1–6

  • Ren S, He K, Girshick R (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In: Proceedings of the 2015 advances in neural information processing systems, pp 91–99

  • Van De Sande KEA, Uijlings JRR, Gevers T, Smeulders AWM (2011) Segmentation as selective search for object recognition. In: Proceedings of the IEEE international conference on computer vision, pp 1879–1886

  • Shi R, Li T, Yamaguchi Y (2020) An attribution-based pruning method for real-time mango detection with YOLO network. Comput Electron Agric. https://doi.org/10.1016/j.compag.2020.105214

    Article  Google Scholar 

  • Sivic J, Zisserman A (2003) Video google: a text retrieval approach to object matching in videos. In: Proceedings of the IEEE international conference on computer vision, pp 1–8

  • Suthamathi V, Chen R-C, Dewi C, Chen L-S (2020) Solving unbounded knapsack problem using evolutionary algorithms with bound constrained strategy. Int J Appl Sci Eng (IJASE) 18(1):1–12

    Google Scholar 

  • Tabernik D, Skocaj D (2020) Deep learning for large-scale traffic-sign detection and recognition. IEEE Trans Intell Transp Syst 21:1427–1440. https://doi.org/10.1109/TITS.2019.2913588

    Article  Google Scholar 

  • Tai S-K, Dewi C, Chen R-C et al (2020) Deep learning for traffic sign recognition based on spatial pyramid pooling with scale analysis. Appl Sci (switzerland) 10:6997. https://doi.org/10.3390/app10196997

    Article  Google Scholar 

  • Tian Y, Yang G, Wang Z et al (2019) Apple detection during different growth stages in orchards using the improved YOLO-V3 model. Comput Electron Agric 157:417–426. https://doi.org/10.1016/j.compag.2019.01.012

    Article  Google Scholar 

  • Timofte R, Prisacariu VA, Gool L Van, Reid I (2011) Combining traffic sign detection with 3D tracking towards better driver assistance. In: Emerging topics in computer vision and its applications, pp 425–446

  • Timofte R, Zimmermann K, Van Gool L (2014) Multi-view traffic sign detection, recognition, and 3D localisation. Mach vis Appl 25:633–647. https://doi.org/10.1007/s00138-011-0391-3

    Article  Google Scholar 

  • Wali SB, Abdullah MA, Hannan MA, et al (2019) Vision-based traffic sign detection and recognition systems: current trends and challenges. Sensors (Switzerland) 1–28

  • Wang C (2018) Research and application of traffic sign detection and recognition based on deep learning. In: Proceedings—2018 international conference on robots and intelligent system, ICRIS 2018, pp 150–152

  • Wang J, Yang J, Yu K, et al (2010) Locality-constrained linear coding for image classification. In: Proceedings of the IEEE computer society conference on computer vision and pattern recognition, pp 3360–3367

  • Wang J, Pan Z, Wang G et al (2018) Spatial pyramid pooling of selective convolutional features for vein recognition. IEEE Access 6:28563–28572. https://doi.org/10.1109/ACCESS.2018.2839720

    Article  Google Scholar 

  • Wang A, Wang M, Jiang K et al (2019) A novel lidar data classification algorithm combined densenet with STN. In: IGARSS 2019—2019 IEEE international geoscience and remote sensing symposium, pp 2483–2486

  • Wang J, Yang Y, Wang T, Simon Sherratt R, Zhang J (2020) Big data service architecture: a survey. J Internet Technol 21(2):393–405

    Google Scholar 

  • Wu F, Jin G, Gao M, et al (2019) Helmet detection based on improved YOLO V3 deep model. In: Proceedings of the 2019 IEEE 16th international conference on networking, sensing and control, ICNSC 2019, pp 363–368

  • Xu L, Yan S, Chen X, Wang P (2019a) Motion recognition algorithm based on deep edge-aware pyramid pooling network in human-computer interaction. IEEE Access 7:163806–163813. https://doi.org/10.1109/ACCESS.2019.2952432

    Article  Google Scholar 

  • Xu X, Jin J, Zhang S et al (2019b) Smart data driven traffic sign detection method based on adaptive color threshold and shape symmetry. Future Gener Comput Syst 94:381–391. https://doi.org/10.1016/j.future.2018.11.027

    Article  Google Scholar 

  • Xu Q, Lin R, Yue H et al (2020) Research on small target detection in driving scenarios based on improved Yolo network. IEEE Access 8:27574–27583. https://doi.org/10.1109/ACCESS.2020.2966328

    Article  Google Scholar 

  • Yang J, Yu K, Gong Y, Huang T (2009) Linear spatial pyramid matching using sparse coding for image classification. In: 2009 IEEE computer society conference on computer vision and pattern recognition workshops, CVPR workshops 2009, pp 1794–1801

  • Yang T, Long X, Sangaiah AK et al (2018) Deep detection network for real-life traffic sign in vehicular networks. Comput Netw 136:95–104. https://doi.org/10.1016/j.comnet.2018.02.026

    Article  Google Scholar 

  • Yang H, Chen L, Chen M et al (2019) Tender tea shoots recognition and positioning for picking robot using improved YOLO-V3 model. IEEE Access 7:180998–181011. https://doi.org/10.1109/ACCESS.2019.2958614

    Article  Google Scholar 

  • Yavuz E, Yazici R, Kasapbasi MC, Bilgin TT (2019) Improving initial flattening of convex-shaped free-form mesh surface patches using a dynamic virtual boundary. Comput Syst Sci Eng. https://doi.org/10.32604/csse.2019.34.339

    Article  Google Scholar 

  • Yu C, He X, Ma H et al (2019a) S-DenseNet: a DenseNet compression model based on convolution grouping strategy using skyline method. IEEE Access 7:183604–183613. https://doi.org/10.1109/ACCESS.2019.2960315

    Article  Google Scholar 

  • Yu C, Liu Z, Liu XJ, et al (2019b) A DenseNet feature-based loop closure method for visual SLAM system. In: IEEE international conference on robotics and biomimetics, ROBIO 2019, pp 258–265

  • Yu L, Xia X, Zhou K (2019c) Traffic sign detection based on visual co-saliency in complex scenes. Appl Intell 49:764–790. https://doi.org/10.1007/s10489-018-1298-8

    Article  Google Scholar 

  • Yuan Y, Xiong Z, Wang Q (2017) An incremental framework for video-based traffic sign detection, tracking, and recognition. IEEE Trans Intell Transp Syst 18:1918–1929. https://doi.org/10.1109/TITS.2016.2614548

    Article  Google Scholar 

  • Zeng M, Xiao N (2019) Effective combination of DenseNet and BiLSTM for keyword spotting. IEEE Access 7:10767–10775. https://doi.org/10.1109/ACCESS.2019.2891838

    Article  Google Scholar 

  • Zhang J, Xie Z, Sun J et al (2020a) A cascaded R-CNN with multiscale attention and imbalanced samples for traffic sign detection. IEEE Access 8:29742–29754. https://doi.org/10.1109/ACCESS.2020.2972338

    Article  Google Scholar 

  • Zhang J, Zhong S, Wang T, Chao HC, Wang J (2020b) Blockchain-based systems and applications: a survey. J Internet Technol 21(1):1–14. https://doi.org/10.3966/160792642020012101001

    Article  Google Scholar 

  • Zhang S, Luo H, Wu Z et al (2020c) Efficient heavy hitters identification over speed traffic streams. Comput Mater Contin. https://doi.org/10.32604/cmc.2020.07496

    Article  Google Scholar 

  • Zhao ZQ, Zheng P, Xu ST, Wu X (2019) Object detection with deep learning: a review. IEEE Trans Neural Netw Learn Syst 30(11):3212–3232. https://doi.org/10.1109/TNNLS.2018.2876865

    Article  Google Scholar 

Download references

Acknowledgements

This paper is supported by the Ministry of Science and Technology, Taiwan. The nos are MOST-110-2927-I-324-50, MOST-110-2221-E-324-010, and MOST-109-2622-E-324 -004, Taiwan. Additionally, this study was partially funded by the EU Horizon 2020 program RISE Project ULTRACEPT under Grant 778062.

Author information

Authors and Affiliations

  1. Department of Information Management, Chaoyang University of Technology, Taichung, Taiwan

    Christine Dewi & Rung-Ching Chen

  2. Faculty of Information Technology, Satya Wacana Christian University, Salatiga, Indonesia

    Christine Dewi

  3. School of Creative Technologies, The University of Portsmouth, Portsmouth, PO1 2UP, UK

    Hui Yu

  4. Department of Mathematics and Computer Science, University of Münster, 48149, Münster, Germany

    Xiaoyi Jiang

Authors
  1. Christine Dewi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rung-Ching Chen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Hui Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaoyi Jiang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rung-Ching Chen.

Ethics declarations

Conflict of interest

The authors declared that they have no conflicts of interest in this work.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Dewi, C., Chen, RC., Yu, H. et al. Robust detection method for improving small traffic sign recognition based on spatial pyramid pooling. J Ambient Intell Human Comput 14, 8135–8152 (2023). https://doi.org/10.1007/s12652-021-03584-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-021-03584-0

Keywords

Search

Navigation