Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Advances, Opportunities and Challenges in AI-enabled Technologies for Autonomous and Connected Vehicles Read chapter Read first chapter

2023 | OriginalPaper | Chapter

Robust AI Driving Strategy for Autonomous Vehicles

Authors : Subramanya Nageshrao, Yousaf Rahman, Vladimir Ivanovic, Mrdjan Jankovic, Eric Tseng, Michael Hafner, Dimitar Filev

Published in: AI-enabled Technologies for Autonomous and Connected Vehicles

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

There has been significant progress in sensing, perception, and localization for automated driving, However, due to the wide spectrum of traffic/road structure scenarios and the long tail distribution of human driver behavior, it has remained an open challenge for an intelligent vehicle to always know how to make and execute the best decision on road given available sensing/perception/localization information. In this chapter, we talk about how artificial intelligence and more specifically, reinforcement learning, can take advantage of operational knowledge and safety reflex to make strategical and tactical decisions. We discuss some challenging problems related to the robustness of reinforcement learning solutions and their implications to the practical design of driving strategies for autonomous vehicles. We focus on automated driving on highway and the integration of reinforcement learning, vehicle motion control, and control barrier function, leading to a robust AI driving strategy that can learn and adapt safely.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Future Technology and Research Trends in Automotive Sensing
next chapter Artificially Intelligent Active Safety Systems
Footnotes
1
*Selected portions reprinted, with permission, from [34], ©2019 IEEE.
 
2
Range R is defined as the distance or gap between the ego and target vehicle, i.e., distance between the ego vehicle’s front bumper and target vehicle’s rear bumper.
 
3
Range rate \(\dot{R}\) is defined as relative speed between the ego and target vehicle.
 
4
Alternatively, the curvature can be computed based on the front wheel angle \(\delta \) as \(\kappa =\delta /(L+K_uV_x)\) where L is the vehicle wheelbase, and \(K_u\) is the vehicle understeer gradient in units of rad-s\(^2\)/m. The understeer gradient is defined as: \(K_u=m_f/C_f-m_r/C_r\) where \(m_f\) and \(m_r\) are the front and rear axle mass, \(C_f\) and \(C_r\) are the front and rear axle cornering coefficients.
 
5
For highway driving nominal value of \(a_1\le 5\,^{\circ }\) hence the error introduce by the small heading offset assumption is \(\le 1\%\).
 
6
Duration is defined as time required to reach path offset relative to the target lane center that is withing the range of nominal lane centering oscillations in range of 0.1–0.2 m. For nominal lane width of 3.4 m, this corresponds to approximately 5% of the lane width or the path offset at the lane change start.
 
7
For lane width of 3.4 m and lane change duration of 6 s, the limits are \(a_{y.max}=0.54\, {\text {m/s}}^2\) and \(j_{max}=0.94\, {\text {m/s}}^3\).
 
Literature
1.
Ackermann J, Bünte T (1999) Robust prevention of limit cycles for robustly decoupled car steering dynamics. Kybernetika 35(1):105–116MathSciNetMATH
2.
Alshiekh M, Bloem R, Ehlers R, Könighofer B, Niekum S, Topcu U (2017) Safe reinforcement learning via shielding. arXiv preprint arXiv:​170808611
3.
Ames AD, Xu X, Grizzle JW, Tabuada P (2016) Control barrier function based quadratic programs for safety critical systems. IEEE Trans Autom Control 62(8):3861–3876MathSciNetCrossRef
4.
Aradi S (2020) Survey of deep reinforcement learning for motion planning of autonomous vehicles. IEEE Trans Intell Transp Syst
5.
Buehler M, Iagnemma K, Singh S (2007) The 2005 DARPA grand challenge: the great robot race, vol 36. Springer, BerlinCrossRef
6.
Buehler M, Iagnemma K, Singh S (2009) The DARPA urban challenge: autonomous vehicles in city traffic, vol 56. Springer, BerlinCrossRef
7.
Chen C, Seff A, Kornhauser A, Xiao J (2015) Deepdriving: Learning affordance for direct perception in autonomous driving. In: 2015 IEEE international conference on computer vision (ICCV). IEEE, pp 2722–2730
8.
Coulter RC (1992) Implementation of the pure pursuit path tracking algorithm. Carnegie-Mellon UNIV Pittsburgh PA Robotics INST, Tech. rep
9.
Duda H (1998) Flight control system design considering rate saturation. Aerosp Sci Technol 2(4):265–275CrossRef
10.
Erdmann J (2014) Lane-changing model in sumo. In: Proceedings of the SUMO2014 modeling mobility with open data, vol 24, pp 77–88
11.
Falcone P, Tufo M, Borrelli F, Asgari J, Tseng HE (2007) A linear time varying model predictive control approach to the integrated vehicle dynamics control problem in autonomous systems. In: 2007 46th IEEE conference on decision and control. IEEE, pp 2980–2985
12.
13.
Garcia J, Fernández F (2012) Safe exploration of state and action spaces in reinforcement learning. J Artif Intell Res 45:515–564MathSciNetCrossRef
14.
Garcıa J, Fernández F (2015) A comprehensive survey on safe reinforcement learning. J Mach Learn Res 16(1):1437–1480MathSciNetMATH
15.
Giersiefer A, Dornhege J, Klein P, Klein C (2019) Driver assistance system. US Patent 20190071126A
16.
Hecker S, Dai D, Van Gool L (2018) End-to-end learning of driving models with surround-view cameras and route planners. In: European conference on computer vision (ECCV)
17.
18.
Jankovic M (2018) Robust control barrier functions for constrained stabilization of nonlinear systems. Automatica 96:359–367MathSciNetCrossRef
19.
Jin IG, Avedisov SS, He CR, Qin WB, Sadeghpour M, Orosz G (2018) Experimental validation of connected automated vehicle design among human-driven vehicles. Transp Res Part C Emerg Technol 91:335–352CrossRef
20.
Kesting A, Treiber M (1999) Helbing D (2007) General lane-changing model mobil for car-following models. Transp Res Rec 1:86–94
21.
Kim E, Kim J, Sunwoo M (2014) Model predictive control strategy for smooth path tracking of autonomous vehicles with steering actuator dynamics. Int J Automot Technol 15(7):1155–1164CrossRef
22.
23.
Kreyszig E (1993) Advanced engineering mathematics. Wiley, New York, pp 482–488MATH
24.
Lee S, Tseng HE (2018) Trajectory planning with shadow trolleys for an autonomous vehicle on bending roads and switchbacks. In: 2018 IEEE intelligent vehicles symposium (IV). IEEE, pp 484–489
25.
Li N, Oyler DW, Zhang M, Yildiz Y, Kolmanovsky I, Girard AR (2017) Game theoretic modeling of driver and vehicle interactions for verification and validation of autonomous vehicle control systems. IEEE Trans Control Syst Technol
26.
Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:​150902971
27.
Lin YC, Hong ZW, Liao YH, Shih ML, Liu MY, Sun M (2017) Tactics of adversarial attack on deep reinforcement learning agents. arXiv preprint arXiv:​170306748
28.
Ma X, Driggs-Campbell K, Kochenderfer MJ (2018) Improved robustness and safety for autonomous vehicle control with adversarial reinforcement learning. In: 2018 IEEE intelligent vehicles symposium (IV). IEEE, pp 1665–1671
29.
Maas AL, Hannun AY, Ng AY (2013) Rectifier nonlinearities improve neural network acoustic models. In: Proceedings of ICML, vol 30, p 3
30.
Minderhoud MM, Bovy PH (2001) Extended time-to-collision measures for road traffic safety assessment. Accid Anal Prev 33(1):89–97CrossRef
31.
Mirchevska B, Pek C, Werling M, Althoff M, Boedecker J (2018) High-level decision making for safe and reasonable autonomous lane changing using reinforcement learning. In: 2018 21st international conference on intelligent transportation systems (ITSC). IEEE, pp 2156–2162
32.
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529CrossRef
33.
Mnih V, Badia AP, Mirza M, Graves A, Lillicrap T, Harley T, Silver D, Kavukcuoglu K (2016) Asynchronous methods for deep reinforcement learning. In: International conference on machine learning, pp 1928–1937
34.
Nageshrao S, Tseng HE, Filev D (2019) Autonomous highway driving using deep reinforcement learning. In: 2019 IEEE international conference on systems man and cybernetics (SMC). IEEE, pp 2326–2331
35.
Nageshrao S, Tseng HE, Filev DP, Baker RL, Cruise C, Daehler L, Mohan S, Kusari A (2020) Vehicle adaptive learning. US Patent 10733510
36.
Ngyuen Q, Sreenath K (2016) Exponential control barrier functions for enforcing high relative-degree safety-critical constraints. In: American control conference, pp 322–328
37.
Papadimitriou I, Tomizuka M (2003) Fast lane changing computations using polynomials. In: Proceedings of the 2003 American control conference. IEEE, vol 1, pp 48–53
38.
Pathak D, Agrawal P, Efros AA, Darrell T (2017) Curiosity-driven exploration by self-supervised prediction. In: International conference on machine learning, PMLR, pp 2778–2787
39.
Prajna S, Jadbabaie A, Pappas GJ (2007) A framework for worst-case and stochastic safety verification using barrier certificates. Trans Autom Control 52(8):1415–1428MathSciNetCrossRef
40.
Rahman Y, Jankovic M, Santillo MA (2021) Driver intent prediction with barrier functions. In: American control conference
41.
Rajamani R (2011) Vehicle dynamics and control. Springer Science & Business Media
42.
43.
Silver D, Schrittwieser J, Simonyan K, Antonoglou I, Huang A, Guez A, Hubert T, Baker L, Lai M, Bolton A et al (2017) Mastering the game of go without human knowledge. Nature 550(7676):354CrossRef
44.
Slotine JJE, Li W et al (1991) Applied nonlinear control, vol 199. Prentice Hall, Englewood CliffsMATH
45.
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction, vol 1. MIT Press, CambridgeMATH
46.
Tamar A, Glassner Y, Mannor S (2015) Optimizing the cvar via sampling. In: Twenty-ninth AAAI conference on artificial intelligence
47.
Thomaz AL, Breazeal C (2008) Teachable robots: understanding human teaching behavior to build more effective robot learners. Artif Intell 172(6–7):716–737CrossRef
48.
Toledo T, Zohar D (2007) Modeling duration of lane changes. Transp Res Rec J Transp Res Board 1999:71–78CrossRef
49.
Treiber M, Kesting A (2013) Traffic flow dynamics. Data, models and simulation. Springer, Berlin
50.
Treiber M, Hennecke A, Helbing D (2000) Congested traffic states in empirical observations and microscopic simulations. Phys Rev E 62(2):1805
51.
Tseng HE, Asgari J, Hrovat D, van Der Jagt P, Cherry A, Neads S (2002) Steering robot for evasive maneuvers-experiment and analysis. IFAC Proc 35(2):79–86CrossRef
52.
Vahidi A, Eskandarian A (2003) Research advances in intelligent collision avoidance and adaptive cruise control. IEEE Trans Intell Transp Syst 4(3):143–153CrossRef
53.
Van Hasselt H, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. AAAI 16:2094–2100
54.
Vezzani G, Gupta A, Natale L, Abbeel P (2019) Learning latent state representation for speeding up exploration. arXiv preprint arXiv:​190512621
55.
Wang Y, Shen D, Teoh EK (2000) Lane detection using spline model. Pattern Recogn Lett 21(8):677–689CrossRef
56.
Werling M, Ziegler J, Kammel S, Thrun S (2010) Optimal trajectory generation for dynamic street scenarios in a frenet frame. In: 2010 IEEE international conference on robotics and automation. IEEE, pp 987–993
57.
Wieland P, Allgöwer F (2007) Constructive safety using control barrier functions. IFAC Proc 40(12):462–467CrossRef
58.
Xiao L, Gao F (2010) A comprehensive review of the development of adaptive cruise control systems. Veh Syst Dyn 48(10):1167–1192CrossRef
59.
Xiao W, Belta C (2019) Control barrier functions for systems with high relative degree. In: IEEE 58th conference on decision and control (CDC), pp 474–479
60.
Xu H, Gao Y, Yu F, Darrell T (2017) End-to-end learning of driving models from large-scale video datasets. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2174–2182
61.
Ye F, Zhang S, Wang P, Chan CY (2021) A survey of deep reinforcement learning algorithms for motion planning and control of autonomous vehicles. arXiv preprint arXiv:​210514218
62.
Zhang M, Li N, Girard A, Kolmanovsky I (2017) A finite state machine based automated driving controller and its stochastic optimization. In: ASME 2017 dynamic systems and control conference, American Society of Mechanical Engineers, pp V002T07A002–V002T07A002
63.
Zhang S, Peng H, Nageshrao S, Tseng E (2019) Discretionary lane change decision making using reinforcement learning with model-based exploration. In: 2019 18th IEEE international conference on machine learning and applications (ICMLA). IEEE, pp 844–850
64.
Zhang S, Peng H, Nageshrao S, Tseng HE (2020) Generating socially acceptable perturbations for efficient evaluation of autonomous vehicles. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops, pp 330–331
Metadata
Title
Robust AI Driving Strategy for Autonomous Vehicles
Authors
Subramanya Nageshrao
Yousaf Rahman
Vladimir Ivanovic
Mrdjan Jankovic
Eric Tseng
Michael Hafner
Dimitar Filev
Copyright Year
2023
Book
AI-enabled Technologies for Autonomous and Connected Vehicles

Print ISBN: 978-3-031-06779-2

Electronic ISBN: 978-3-031-06780-8

Copyright Year: 2023

DOI
https://doi.org/10.1007/978-3-031-06780-8_7

Premium Partner

    Image Credits