Skip to main content
Top

2023 | OriginalPaper | Chapter

Unseen Obstacle Detection via Monocular Camera Against Speed Change and Background Noise

Authors : Kai Wang, Siming Lu, Shenlu Jiang

Published in: HCI International 2023 – Late Breaking Papers

Publisher: Springer Nature Switzerland

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This paper proposes a novel obstacle detection system optimized for mobile platforms. The system uses a long-short step Recurrent Neural Network (RNN) for optical flow estimation in various speed scenarios, combined with a global direction filter that filters out background noise, resulting in more robust and accurate obstacle detection. The system is designed to maximize processing speed and resource efficiency on mobile platforms. Performance is demonstrated on own-collected YouTube videos, achieving a precision rate of 95.2%, recall rate of 94.3%, and a frame rate of 75 FPS, surpassing state-of-the-art optical flow techniques. The proposed method is also evaluated on a real robot platform, demonstrating robust performance in detecting and avoiding obstacles of varying sizes and speeds under different lighting and noise conditions. Overall, the proposed system offers a reliable and efficient solution for obstacle detection and avoidance on mobile platforms, with high confidence in obstacle detection and avoidance in various real-world scenarios.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Jiang, S., Hong, Z.: Unexpected dynamic obstacle monocular detection in the driver view. IEEE Intell. Transp. Syst. Mag. 14(1), 65–74 (2022)MathSciNet Jiang, S., Hong, Z.: Unexpected dynamic obstacle monocular detection in the driver view. IEEE Intell. Transp. Syst. Mag. 14(1), 65–74 (2022)MathSciNet
2.
go back to reference Siegwart, R., Nourbakhsh, I.R., Scaramuzza, D.: Introduction to Autonomous Robots: Mechanisms and Sensors. MIT Press, Cambridge (2011) Siegwart, R., Nourbakhsh, I.R., Scaramuzza, D.: Introduction to Autonomous Robots: Mechanisms and Sensors. MIT Press, Cambridge (2011)
3.
go back to reference Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015) Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Advances in Neural Information Processing Systems, pp. 91–99 (2015)
4.
go back to reference Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015) Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
5.
go back to reference Esfahlani, S.: Shabnam: the deep convolutional neural network role in the autonomous navigation of mobile robots (SROBO). Remote Sens. 14, 3324–3324 (2022)CrossRef Esfahlani, S.: Shabnam: the deep convolutional neural network role in the autonomous navigation of mobile robots (SROBO). Remote Sens. 14, 3324–3324 (2022)CrossRef
6.
go back to reference Li, Y., Liu, M., Tang, J.: Robust obstacle detection based on optical flow and SVM. J. Comput. Sci. Technol. 29(1), 52–61 (2014) Li, Y., Liu, M., Tang, J.: Robust obstacle detection based on optical flow and SVM. J. Comput. Sci. Technol. 29(1), 52–61 (2014)
7.
go back to reference Khan, A., Hamza, S., Li, X.: Luo: Obstacle avoidance and tracking control of redundant robotic manipulator: an RNN-based metaheuristic approach. IEEE Trans. Industr. Inf. 16, 4670–4680 (2019)CrossRef Khan, A., Hamza, S., Li, X.: Luo: Obstacle avoidance and tracking control of redundant robotic manipulator: an RNN-based metaheuristic approach. IEEE Trans. Industr. Inf. 16, 4670–4680 (2019)CrossRef
8.
go back to reference Tian, L., Tu, Z., Zhang, D., Liu, J., Li, B., Yuan, J.: Unsupervised learning of optical flow with CNN-based non-local filtering. IEEE Trans. Image Process. 29, 8429–8442 (2020)CrossRefMATH Tian, L., Tu, Z., Zhang, D., Liu, J., Li, B., Yuan, J.: Unsupervised learning of optical flow with CNN-based non-local filtering. IEEE Trans. Image Process. 29, 8429–8442 (2020)CrossRefMATH
9.
go back to reference Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRef Hirschmuller, H.: Stereo processing by semiglobal matching and mutual information. IEEE Trans. Pattern Anal. Mach. Intell. 30(2), 328–341 (2008)CrossRef
10.
go back to reference Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47(1–3), 7–42 (2002)CrossRefMATH Scharstein, D., Szeliski, R.: A taxonomy and evaluation of dense two-frame stereo correspondence algorithms. Int. J. Comput. Vision 47(1–3), 7–42 (2002)CrossRefMATH
11.
go back to reference Chuah, W., Tennakoon, R., Hoseinnezhad, R., Suter, D., Bab-Hadiashar, A.: Semantic guided long range stereo depth estimation for safer autonomous vehicle applications. IEEE Trans. Intell. Transp. Syst. 23(10), 18916–18926 (2022)CrossRef Chuah, W., Tennakoon, R., Hoseinnezhad, R., Suter, D., Bab-Hadiashar, A.: Semantic guided long range stereo depth estimation for safer autonomous vehicle applications. IEEE Trans. Intell. Transp. Syst. 23(10), 18916–18926 (2022)CrossRef
12.
go back to reference Dodge, D., Yilmaz, M.: Convex vision-based negative obstacle detection framework for autonomous vehicles. IEEE Trans. Intell. Veh. 8(1), 778–789 (2022)CrossRef Dodge, D., Yilmaz, M.: Convex vision-based negative obstacle detection framework for autonomous vehicles. IEEE Trans. Intell. Veh. 8(1), 778–789 (2022)CrossRef
13.
go back to reference Chen, G., Peng, P., Zhang, P., Dong, W.: Risk-aware trajectory sampling for quadrotor obstacle avoidance in dynamic environments. IEEE Trans. Ind. Electron. (2023) Chen, G., Peng, P., Zhang, P., Dong, W.: Risk-aware trajectory sampling for quadrotor obstacle avoidance in dynamic environments. IEEE Trans. Ind. Electron. (2023)
14.
go back to reference Xu, Z., Zhou, X., Wu, H., Li, X., Li, S.: Motion planning of manipulators for simultaneous obstacle avoidance and target tracking: an RNN approach with guaranteed performance. IEEE Trans. Industr. Electron. 69(4), 3887–3897 (2021)CrossRef Xu, Z., Zhou, X., Wu, H., Li, X., Li, S.: Motion planning of manipulators for simultaneous obstacle avoidance and target tracking: an RNN approach with guaranteed performance. IEEE Trans. Industr. Electron. 69(4), 3887–3897 (2021)CrossRef
15.
go back to reference Perumal, P.S., et al.: An insight into crash avoidance and overtaking advice systems for autonomous vehicles: a review, challenges and solutions. Eng. Appl. Artif. Intell. 104, 104406 (2021)CrossRef Perumal, P.S., et al.: An insight into crash avoidance and overtaking advice systems for autonomous vehicles: a review, challenges and solutions. Eng. Appl. Artif. Intell. 104, 104406 (2021)CrossRef
16.
go back to reference Oroko, J.A., Nyakoe, G.N.: Obstacle avoidance and path planning schemes for autonomous navigation of a mobile robot: a review. In: Proceedings of the Sustainable Research and Innovation Conference (2022) Oroko, J.A., Nyakoe, G.N.: Obstacle avoidance and path planning schemes for autonomous navigation of a mobile robot: a review. In: Proceedings of the Sustainable Research and Innovation Conference (2022)
17.
go back to reference Hutabarat, D., Rivai, M., Purwanto, D., Hutomo, H.: Lidar-based obstacle avoidance for the autonomous mobile robot. In: 12th International Conference on Information & Communication Technology and System (ICTS), pp. 197–202 (2019) Hutabarat, D., Rivai, M., Purwanto, D., Hutomo, H.: Lidar-based obstacle avoidance for the autonomous mobile robot. In: 12th International Conference on Information & Communication Technology and System (ICTS), pp. 197–202 (2019)
18.
go back to reference Chen, K.: Mvlidarnet: real-time multi-class scene understanding for autonomous driving using multiple views. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020) Chen, K.: Mvlidarnet: real-time multi-class scene understanding for autonomous driving using multiple views. In: 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) (2020)
19.
go back to reference Wang, P., Gao, S., Li, L., Sun, B., Cheng, S.: Obstacle avoidance path planning design for autonomous driving vehicles based on an improved artificial potential field algorithm. Energies 12(12), 2342–2342 (2019)CrossRef Wang, P., Gao, S., Li, L., Sun, B., Cheng, S.: Obstacle avoidance path planning design for autonomous driving vehicles based on an improved artificial potential field algorithm. Energies 12(12), 2342–2342 (2019)CrossRef
20.
go back to reference Pandey, A., Kashyap, A.K., Parhi, D.R., Patle, B.K.: Autonomous mobile robot navigation between static and dynamic obstacles using multiple ANFIS architecture. World J. Eng. 16(2), 275–286 (2019)CrossRef Pandey, A., Kashyap, A.K., Parhi, D.R., Patle, B.K.: Autonomous mobile robot navigation between static and dynamic obstacles using multiple ANFIS architecture. World J. Eng. 16(2), 275–286 (2019)CrossRef
21.
go back to reference Zhao, X., Sun, P., Xu, Z., Min, H., Yu, H.: Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications. IEEE Sens. J. 20(9), 4901–4913 (2020)CrossRef Zhao, X., Sun, P., Xu, Z., Min, H., Yu, H.: Fusion of 3D LIDAR and camera data for object detection in autonomous vehicle applications. IEEE Sens. J. 20(9), 4901–4913 (2020)CrossRef
22.
go back to reference Hua, M., Nan, Y., Lian, S.: Small obstacle avoidance based on RGB-D semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019) Hua, M., Nan, Y., Lian, S.: Small obstacle avoidance based on RGB-D semantic segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision Workshops (2019)
23.
go back to reference Zhao, C., Sun, Q., Zhang, C., Tang, Y., Qian, F.: Monocular depth estimation based on deep learning: an overview. SCIENCE CHINA Technol. Sci. 63(9), 1612–1627 (2020)CrossRef Zhao, C., Sun, Q., Zhang, C., Tang, Y., Qian, F.: Monocular depth estimation based on deep learning: an overview. SCIENCE CHINA Technol. Sci. 63(9), 1612–1627 (2020)CrossRef
24.
go back to reference Croon, D., Guido, C., De Wagter, C., Seidl, T.: Enhancing optical-flow-based control by learning visual appearance cues for flying robots. Nat. Mach. Intell. 3(1), 33–41 (2021)CrossRef Croon, D., Guido, C., De Wagter, C., Seidl, T.: Enhancing optical-flow-based control by learning visual appearance cues for flying robots. Nat. Mach. Intell. 3(1), 33–41 (2021)CrossRef
25.
go back to reference Fox, W., Burgard, S.: Thrun: the dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 4(1), 23–33 (1997)CrossRef Fox, W., Burgard, S.: Thrun: the dynamic window approach to collision avoidance. IEEE Robot. Autom. Mag. 4(1), 23–33 (1997)CrossRef
26.
go back to reference Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vision 56, 221–255 (2004)CrossRefMATH Baker, S., Matthews, I.: Lucas-kanade 20 years on: a unifying framework. Int. J. Comput. Vision 56, 221–255 (2004)CrossRefMATH
27.
go back to reference Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470 (2017) Ilg, E., Mayer, N., Saikia, T., Keuper, M., Dosovitskiy, A., Brox, T.: Flownet 2.0: evolution of optical flow estimation with deep networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2462–2470 (2017)
Metadata
Title
Unseen Obstacle Detection via Monocular Camera Against Speed Change and Background Noise
Authors
Kai Wang
Siming Lu
Shenlu Jiang
Copyright Year
2023
DOI
https://doi.org/10.1007/978-3-031-48047-8_42