Skip to main content
Top
Published in:

31-10-2023

A Novel Cognitively Inspired Deep Learning Approach to Detect Drivable Areas for Self-driving Cars

Authors: Fengling Jiang, Zeling Wang, Guoqing Yue

Published in: Cognitive Computation | Issue 2/2024

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Road drivable area detection is an important task in computer vision with applications in self-driving cars. Accurately detecting and mapping drivable areas in a scene allow vehicles and robots to plan safe trajectories. In this paper, a novel cognitively inspired approach is proposed that considers both the salient areas in a driving scene and the driver’s attention mechanism. Specifically, the attention point is computed by combining salient areas and attention regions. Furthermore, we use the attention point and two boundary nodes on the road edge to form a triangular road surface area. Finally, we segment this area and remove the salient region within this area to obtain the drivable road area. Experimental results show that our proposed method can address the shortcomings of traditional vanishing point detection algorithms and enhance drivable area perception when combined with 4 different backbones on the DeepLabV3+ model. In particular, we demonstrate the effectiveness of merging salient area and attention area algorithms and explore the joint understanding of these complementary visual cues.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Hu Y, Yang J, Chen L, Li K, Sima C, Zhu X, Chai S, Du S, Lin T, Wang W et al. Planning-oriented autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2023. p. 17853–62. Hu Y, Yang J, Chen L, Li K, Sima C, Zhu X, Chai S, Du S, Lin T, Wang W et al. Planning-oriented autonomous driving. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2023. p. 17853–62.
2.
go back to reference Xiong X, Liu Y, Yuan T, Wang Y, Wang Y, Zhao H. Neural map prior for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2023. p. 17535–44. Xiong X, Liu Y, Yuan T, Wang Y, Wang Y, Zhao H. Neural map prior for autonomous driving. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE; 2023. p. 17535–44.
3.
go back to reference Acun O, Küçükmanisa A, Genç Y, Urhan O. D3net (divide and detect drivable area net): Deep learning based drivable area detection and its embedded application. J Real-Time Image Proc. 2023;20(2):16.CrossRef Acun O, Küçükmanisa A, Genç Y, Urhan O. D3net (divide and detect drivable area net): Deep learning based drivable area detection and its embedded application. J Real-Time Image Proc. 2023;20(2):16.CrossRef
4.
go back to reference Zhong C, Li B, Tao W. Off-road drivable area detection: A learning-based approach exploiting lidar reflection texture information. Remote Sens. 2022;15(1):27.CrossRef Zhong C, Li B, Tao W. Off-road drivable area detection: A learning-based approach exploiting lidar reflection texture information. Remote Sens. 2022;15(1):27.CrossRef
5.
go back to reference Qiao D, Zulkernine F. Drivable area detection using deep learning models for autonomous driving. In: Proceedings of the IEEE International Conference on Big Data. IEEE; 2021. p. 5233–8. Qiao D, Zulkernine F. Drivable area detection using deep learning models for autonomous driving. In: Proceedings of the IEEE International Conference on Big Data. IEEE; 2021. p. 5233–8.
6.
go back to reference Maddern W, Pascoe G, Linegar C, Newman P. 1 year, 1000 km: The oxford robotcar dataset. Int J Robot Res. 2017;36(1):3–15.CrossRef Maddern W, Pascoe G, Linegar C, Newman P. 1 year, 1000 km: The oxford robotcar dataset. Int J Robot Res. 2017;36(1):3–15.CrossRef
7.
go back to reference Kong H, Audibert J-Y, Ponce J. Vanishing point detection for road detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009. p. 96–103. Kong H, Audibert J-Y, Ponce J. Vanishing point detection for road detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009. p. 96–103.
8.
go back to reference Moghadam P, Starzyk JA, Wijesoma WS. Fast vanishing-point detection in unstructured environments. IEEE Trans Image Process. 2011;21(1):425–30. Moghadam P, Starzyk JA, Wijesoma WS. Fast vanishing-point detection in unstructured environments. IEEE Trans Image Process. 2011;21(1):425–30.
9.
go back to reference Wang G, Zhang B, Wang H, Lin X, Li Y, Liu Z. Detection of the drivable area on high-speed road via yolact. SIViP. 2022;16(6):1623–30.CrossRef Wang G, Zhang B, Wang H, Lin X, Li Y, Liu Z. Detection of the drivable area on high-speed road via yolact. SIViP. 2022;16(6):1623–30.CrossRef
10.
go back to reference Jiqing C, Depeng W, Teng L, Tian L, Huabin W. All-weather road drivable area segmentation method based on cyclegan. Vis Comput. 2022;39:1–17. Jiqing C, Depeng W, Teng L, Tian L, Huabin W. All-weather road drivable area segmentation method based on cyclegan. Vis Comput. 2022;39:1–17.
11.
go back to reference Ayub A, Wagner AR. Cbcl-pr: A cognitively inspired model for class-incremental learning in robotics. IEEE Trans Cognit Develop Syst. 2023;1–1. Ayub A, Wagner AR. Cbcl-pr: A cognitively inspired model for class-incremental learning in robotics. IEEE Trans Cognit Develop Syst. 2023;1–1.
12.
go back to reference Wang X, Liu J, Mei T, Luo J. Coseg: Cognitively inspired unsupervised generic event segmentation. IEEE Trans Neural Netw Learn Syst. 2023;1–11. Wang X, Liu J, Mei T, Luo J. Coseg: Cognitively inspired unsupervised generic event segmentation. IEEE Trans Neural Netw Learn Syst. 2023;1–11.
13.
go back to reference Fei GAO, Aidong LIU, Kai LIU, Erfu YANG, Hussain A. A novel visual attention method for target detection from SAR images. Chinese J Aeronaut. 2019;32(8):1946–58.CrossRef Fei GAO, Aidong LIU, Kai LIU, Erfu YANG, Hussain A. A novel visual attention method for target detection from SAR images. Chinese J Aeronaut. 2019;32(8):1946–58.CrossRef
14.
go back to reference Rolls ET. Brain computations and connectivity. Oxford University Press. 2023. Rolls ET. Brain computations and connectivity. Oxford University Press. 2023.
15.
go back to reference Mochizuki-Freeman J, Maini SS, Tiganj Z. Characterizing neural activity in cognitively inspired RL agents during an evidence accumulation task. In: Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN). IEEE; 2023. p. 1–9. Mochizuki-Freeman J, Maini SS, Tiganj Z. Characterizing neural activity in cognitively inspired RL agents during an evidence accumulation task. In: Proceedings of the IEEE International Joint Conference on Neural Networks (IJCNN). IEEE; 2023. p. 1–9.
16.
go back to reference Wichmann FA, Geirhos R. Are deep neural networks adequate behavioral models of human visual perception? Ann Rev Vis Sci. 2023;9:501–24.CrossRef Wichmann FA, Geirhos R. Are deep neural networks adequate behavioral models of human visual perception? Ann Rev Vis Sci. 2023;9:501–24.CrossRef
17.
go back to reference Räuker T, Ho A, Casper S, Hadfield-Menell D. Toward transparent AI: A survey on interpreting the inner structures of deep neural networks. In: Proceedings of the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE; 2023. p. 464–83. Räuker T, Ho A, Casper S, Hadfield-Menell D. Toward transparent AI: A survey on interpreting the inner structures of deep neural networks. In: Proceedings of the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE; 2023. p. 464–83.
18.
go back to reference Fuchs A, Passarella A, Conti M. A cognitive framework for delegation between error-prone AI and human agents. In: Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP). IEEE; 2022. p. 317–22. Fuchs A, Passarella A, Conti M. A cognitive framework for delegation between error-prone AI and human agents. In: Proceedings of the IEEE International Conference on Smart Computing (SMARTCOMP). IEEE; 2022. p. 317–22.
19.
go back to reference Ko M, Higgins L, Chrysler ST, Lord D. Effect of driving environment on drivers’ eye movements: Re-analyzing previously collected eye-tracker data. Technical report. 2010. Ko M, Higgins L, Chrysler ST, Lord D. Effect of driving environment on drivers’ eye movements: Re-analyzing previously collected eye-tracker data. Technical report. 2010.
20.
go back to reference Deng T, Yang K, Li Y, Yan H. Where does the driver look? Top-down-based saliency detection in a traffic driving environment. IEEE Trans Intell Transp Syst. 2016;17(7):2051–62.CrossRef Deng T, Yang K, Li Y, Yan H. Where does the driver look? Top-down-based saliency detection in a traffic driving environment. IEEE Trans Intell Transp Syst. 2016;17(7):2051–62.CrossRef
21.
go back to reference Moghadam P, Dong JF. Road direction detection based on vanishing-point tracking. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE; 2012. p. 1553–60. Moghadam P, Dong JF. Road direction detection based on vanishing-point tracking. In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems. IEEE; 2012. p. 1553–60.
22.
go back to reference Zhou S, Jiang Y, Xi J, Gong J, Xiong G, Chen H. A novel lane detection based on geometrical model and Gabor filter. In: Proceedings of the IEEE Intelligent Vehicles Symposium. IEEE; 2010. p. 59–64. Zhou S, Jiang Y, Xi J, Gong J, Xiong G, Chen H. A novel lane detection based on geometrical model and Gabor filter. In: Proceedings of the IEEE Intelligent Vehicles Symposium. IEEE; 2010. p. 59–64.
23.
go back to reference Ziyu C, Zhen H. Simple road detection based on vanishing point. J Electron Imaging. 2014;23(3):033015–033015.CrossRef Ziyu C, Zhen H. Simple road detection based on vanishing point. J Electron Imaging. 2014;23(3):033015–033015.CrossRef
24.
go back to reference Li Y, Ding W, Zhang X, Zhaojie J. Road detection algorithm for autonomous navigation systems based on dark channel prior and vanishing point in complex road scenes. Robot Auton Syst. 2016;85:1–11.CrossRef Li Y, Ding W, Zhang X, Zhaojie J. Road detection algorithm for autonomous navigation systems based on dark channel prior and vanishing point in complex road scenes. Robot Auton Syst. 2016;85:1–11.CrossRef
25.
go back to reference Hirschmüller H, Innocent PR, Garibaldi J. Real-time correlation-based stereo vision with reduced border errors. Int J Comput Vis. 2002;47:229–46.CrossRef Hirschmüller H, Innocent PR, Garibaldi J. Real-time correlation-based stereo vision with reduced border errors. Int J Comput Vis. 2002;47:229–46.CrossRef
26.
go back to reference Mogelmose A, Trivedi MM, Moeslund TB. Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey. IEEE Trans Intell Transport Syst. 2012;13(4):1484–97.CrossRef Mogelmose A, Trivedi MM, Moeslund TB. Vision-based traffic sign detection and analysis for intelligent driver assistance systems: Perspectives and survey. IEEE Trans Intell Transport Syst. 2012;13(4):1484–97.CrossRef
27.
go back to reference Fritsch J, Kuehnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms. In: Proceedings of the IEEE Conference on Intelligent Transportation Systems. IEEE; 2013. p. 1693–1700. Fritsch J, Kuehnl T, Geiger A. A new performance measure and evaluation benchmark for road detection algorithms. In: Proceedings of the IEEE Conference on Intelligent Transportation Systems. IEEE; 2013. p. 1693–1700.
28.
go back to reference Fan C, Jingbo X, Di S. Lane detection based on machine learning algorithm. TELKOMNIKA Indones J Electr Eng. 2014;12(2):1403–9. Fan C, Jingbo X, Di S. Lane detection based on machine learning algorithm. TELKOMNIKA Indones J Electr Eng. 2014;12(2):1403–9.
29.
go back to reference Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2015. p. 3431–40. Long J, Shelhamer E, Darrell T. Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2015. p. 3431–40.
30.
go back to reference He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Proceedings of the European Conference on Computer Vision. Springer; 2016. p. 630–45. He K, Zhang X, Ren S, Sun J. Identity mappings in deep residual networks. In: Proceedings of the European Conference on Computer Vision. Springer; 2016. p. 630–45.
31.
go back to reference Zhu Y, Newsam S. Densenet for dense flow. In: Proceedings of the IEEE International Conference on Image Processing. IEEE; 2017. p. 790–4. Zhu Y, Newsam S. Densenet for dense flow. In: Proceedings of the IEEE International Conference on Image Processing. IEEE; 2017. p. 790–4.
32.
go back to reference Tang J, Li S, Liu P. A review of lane detection methods based on deep learning. Pattern Recogn. 2021;111:107623.CrossRef Tang J, Li S, Liu P. A review of lane detection methods based on deep learning. Pattern Recogn. 2021;111:107623.CrossRef
33.
go back to reference Wang H, Fan R, Sun Y, Liu M. Dynamic fusion module evolves drivable area and road anomaly detection: A benchmark and algorithms. IEEE Trans Cybern. 2021;52(10):10750–60.CrossRef Wang H, Fan R, Sun Y, Liu M. Dynamic fusion module evolves drivable area and road anomaly detection: A benchmark and algorithms. IEEE Trans Cybern. 2021;52(10):10750–60.CrossRef
34.
go back to reference Rawashdeh NA, Bos JP, Abu-Alrub NJ. Camera-lidar sensor fusion for drivable area detection in winter weather using convolutional neural networks. Opt Eng. 2023;62(3):031202–031202. Rawashdeh NA, Bos JP, Abu-Alrub NJ. Camera-lidar sensor fusion for drivable area detection in winter weather using convolutional neural networks. Opt Eng. 2023;62(3):031202–031202.
35.
go back to reference Li Y, Ren J, Yan Y, Liu Q, Ma P, Petrovski A, Sun H. Cbanet: An end-to-end cross band 2-d attention network for hyperspectral change detection in remote sensing. IEEE Trans Geosci Remote Sens. 2023;61:1–11. Li Y, Ren J, Yan Y, Liu Q, Ma P, Petrovski A, Sun H. Cbanet: An end-to-end cross band 2-d attention network for hyperspectral change detection in remote sensing. IEEE Trans Geosci Remote Sens. 2023;61:1–11.
36.
go back to reference Li H, Tianyang X, Xiao-Jun W, Jiwen L, Kittler J. Lrrnet: A novel representation learning guided fusion network for infrared and visible images. IEEE Trans Pattern Anal Mach Intell. 2023;45(9):11040–52.CrossRef Li H, Tianyang X, Xiao-Jun W, Jiwen L, Kittler J. Lrrnet: A novel representation learning guided fusion network for infrared and visible images. IEEE Trans Pattern Anal Mach Intell. 2023;45(9):11040–52.CrossRef
37.
go back to reference Ying Z, Li G, Zang X, Wang R, Wang W. A novel shadow-free feature extractor for real-time road detection. In: Proceedings of the ACM International Conference on Multimedia. ACM; 2016. p. 611–5. Ying Z, Li G, Zang X, Wang R, Wang W. A novel shadow-free feature extractor for real-time road detection. In: Proceedings of the ACM International Conference on Multimedia. ACM; 2016. p. 611–5.
38.
go back to reference Illingworth J, Kittler J. The adaptive hough transform. IEEE Trans Pattern Anal Mach Intell. 1987;PAMI-9(5):690–8. Illingworth J, Kittler J. The adaptive hough transform. IEEE Trans Pattern Anal Mach Intell. 1987;PAMI-9(5):690–8.
39.
go back to reference Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009. p. 1597–604. Achanta R, Hemami S, Estrada F, Susstrunk S. Frequency-tuned salient region detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009. p. 1597–604.
40.
go back to reference Jiang F, Kong B, Li J, Dashtipour K, Gogate M. Robust visual saliency optimization based on bidirectional Markov chains. Cogn Comput. 2021;13:69–80.CrossRef Jiang F, Kong B, Li J, Dashtipour K, Gogate M. Robust visual saliency optimization based on bidirectional Markov chains. Cogn Comput. 2021;13:69–80.CrossRef
41.
go back to reference Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern. 1979;9(1):62–6.CrossRef Otsu N. A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybern. 1979;9(1):62–6.CrossRef
42.
go back to reference Liu F, Fang M. Semantic segmentation of underwater images based on improved deeplab. J Mar Sci Eng. 2020;8(3):188.CrossRef Liu F, Fang M. Semantic segmentation of underwater images based on improved deeplab. J Mar Sci Eng. 2020;8(3):188.CrossRef
43.
go back to reference Bui TH, Nobuyama E, Saitoh T. A texture-based local soft voting method for vanishing point detection from a single road image. IEICE Trans Inf Syst. 2013;96(3):690–8.CrossRef Bui TH, Nobuyama E, Saitoh T. A texture-based local soft voting method for vanishing point detection from a single road image. IEICE Trans Inf Syst. 2013;96(3):690–8.CrossRef
44.
go back to reference Ying Z, Li G, Tan G. An illumination-robust approach for feature-based road detection. In: Proceedings of the IEEE International Symposium on Multimedia. IEEE; 2015. p. 278–81. Ying Z, Li G, Tan G. An illumination-robust approach for feature-based road detection. In: Proceedings of the IEEE International Symposium on Multimedia. IEEE; 2015. p. 278–81.
45.
go back to reference Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision. Springer; 2018. p. 801–18. Chen L-C, Zhu Y, Papandreou G, Schroff F, Adam H. Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Proceedings of the European Conference on Computer Vision. Springer; 2018. p. 801–18.
46.
go back to reference Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 [Preprint]. 2017. Available from: http://arxiv.org/abs/1704.04861. Howard AG, Zhu M, Chen B, Kalenichenko D, Wang W, Weyand T, Andreetto M, Adam H. Mobilenets: Efficient convolutional neural networks for mobile vision applications. arXiv:1704.04861 [Preprint]. 2017. Available from: http://​arxiv.​org/​abs/​1704.​04861.
47.
go back to reference Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2017. p. 1251–8. Chollet F. Xception: Deep learning with depthwise separable convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2017. p. 1251–8.
48.
go back to reference Yu F, Koltun V, Funkhouser T. Dilated residual networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2017. p. 472–80. Yu F, Koltun V, Funkhouser T. Dilated residual networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2017. p. 472–80.
Metadata
Title
A Novel Cognitively Inspired Deep Learning Approach to Detect Drivable Areas for Self-driving Cars
Authors
Fengling Jiang
Zeling Wang
Guoqing Yue
Publication date
31-10-2023
Publisher
Springer US
Published in
Cognitive Computation / Issue 2/2024
Print ISSN: 1866-9956
Electronic ISSN: 1866-9964
DOI
https://doi.org/10.1007/s12559-023-10215-7

Other articles of this Issue 2/2024

Cognitive Computation 2/2024 Go to the issue

Premium Partner