Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Advances, Opportunities and Challenges in AI-enabled Technologies for Autonomous and Connected Vehicles Read chapter Read first chapter

2023 | OriginalPaper | Chapter

Self-learning Decision and Control for Highly Automated Vehicles

Authors : Jianyu Chen, Jingliang Duan, Yang Guan, Qi Sun, Yuming Yin, Shengbo Eben Li

Published in: AI-enabled Technologies for Autonomous and Connected Vehicles

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The decision and control module plays a key role for autonomous driving, which is responsible for generating appropriate control commands that navigate the autonomous vehicles safely and efficiently. Existing decision and control modules for automated vehicles are mainly using a rule-based hand-engineered approach. Although working well in a number of specialized scenarios, such method shows its limitation when dealing with highly automated driving tasks such as dense urban scenarios. Recent advances in artificial intelligence have inspired a line of works about self-learning based decision and control, which enable self-reinforcement of the control policy to potentially super-human performance. In this chapter, we will introduce how to appropriately apply such techniques to automated vehicles. The chapter will begin with the motivations and basics, followed by the key challenges and recent achievements of self-learning decision and control for automated vehicles, focusing on the following key aspects: scalability, performance, interpretability, mixed-model, and emergency handling.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Energy-Efficient Autonomous Driving Using Cognitive Driver Behavioral Models and Reinforcement Learning
next chapter MAGMA: Mobility Analytics Generated from Metrics on ADAS
Literature
1.
Bojarski M, Yeres P, Choromanska A, Choromanski K, Firner B, Jackel L, Muller U (2017) Explaining how a deep neural network trained with end-to-end learning steers a car. arXiv preprint arXiv:​1704.​07911
2.
Chen J, Li SE, Tomizuka M (2021) Interpretable end-to-end urban autonomous driving with latent deep reinforcement learning. IEEE Trans Intell Transp Syst
3.
Chen J, Wang Z, Tomizuka M (2018) Deep hierarchical reinforcement learning for autonomous driving with distinct behaviors. In: 2018 IEEE intelligent vehicles symposium (IV). IEEE, pp 1239–1244
4.
Chen J, Yuan B, Tomizuka M (2019) Model-free deep reinforcement learning for urban autonomous driving. In: 2019 IEEE intelligent transportation systems conference (ITSC). IEEE, pp 2765–2771
5.
Duan J, Guan Y, Li SE, Ren Y, Sun Q, Cheng B (2021) Distributional soft actor-critic: off-policy reinforcement learning for addressing value estimation errors. IEEE Trans Neural Netw Learn Syst
6.
Duan J, Li SE, Guan Y, Sun Q, Cheng B (2020) Hierarchical reinforcement learning for self-driving decision-making without reliance on labelled driving data. IET Intell Transp Syst 14(5):297–305
7.
Duan J, Yu D, Li SE, Wang W, Ren Y, Lin Z, Cheng B (2021) Fixed-dimensional and permutation invariant state representation of autonomous driving. arXiv preprint arXiv:​2105.​11299
8.
Emuna R, Borowsky A, Biess A (2020) Deep reinforcement learning for human-like driving policies in collision avoidance tasks of self-driving cars. arXiv preprint arXiv:​2006.​04218
9.
Finn C, Abbeel P, Levine S (2017) Model-agnostic meta-learning for fast adaptation of deep networks. In: International conference on machine learning. PMLR, pp 1126–1135
10.
Fujimoto S, van Hoof H, Meger D (2018) Addressing function approximation error in actor-critic methods. arXiv preprint arXiv:​1802.​09477
11.
Gu Z, Yang Y, Duan J, Li SE, Chen J, Cao W, Zheng S (2021) Belief state separated reinforcement learning for autonomous vehicle decision making under uncertainty. In: 2021 IEEE 24th international conference on intelligent transportation systems (ITSC), pp 1–7
12.
Guan Y, Li SE, Duan J, Li J, Ren Y, Sun Q, Cheng B (2019) Direct and indirect reinforcement learning. Int J Intell Syst
13.
Guan Y, Li SE, Duan J, Wang W, Cheng B (2018) Markov probabilistic decision making of self-driving cars in highway with random traffic flow: a simulation study. J Intell Connected Veh
14.
Guan Y, Ren Y, Li SE, Sun Q, Luo L, Li K (2020) Centralized cooperation for connected and automated vehicles at intersections by proximal policy optimization. IEEE Trans Veh Technol 69(11):12597–12608
15.
Guan Y, Ren Y, Sun Q, Li SE, Ma H, Duan J, Dai Y, Cheng B (2021) Integrated decision and control: towards interpretable and computationally efficient driving intelligence. arXiv preprint arXiv:​2103.​10290
16.
Guo J, Kurup U, Shah Mohak (2019) Is it safe to drive? An overview of factors, metrics, and datasets for driveability assessment in autonomous driving. IEEE Trans Intell Transp Syst 21(8):3135–3151CrossRef
17.
Haarnoja T, Tang H, Abbeel P, Levine S (2017) Reinforcement learning with deep energy-based policies. In: Proceedings of the 34th international conference on machine learning, vol 70, pp 1352–1361. (JMLR-organization)
18.
Haarnoja T, Zhou A, Abbeel P, Levine S (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. arXiv preprint arXiv:​1801.​01290
19.
Haarnoja T, Zhou A, Hartikainen K, Tucker G, Ha S, Tan J, Kumar V, Zhu H, Gupta A, Abbeel P et al (2018) Soft actor-critic algorithms and applications. arXiv preprint arXiv:​1812.​05905
20.
Hafner D, Lillicrap T, Fischer I, Villegas R, Ha D, Lee H, Davidson J (2018) Learning latent dynamics for planning from pixels. arXiv preprint arXiv:​1811.​04551
21.
Hafner D, Lillicrap T, Fischer I, Villegas R, Ha D, Lee H, Davidson J (2019) Learning latent dynamics for planning from pixels. In: International conference on machine learning, pp 2555–2565. PMLR
22.
Hou L, Xin L, Li SE, Cheng B, Wang W (2019) Interactive trajectory prediction of surrounding road users for autonomous driving using structural-LSTM network. IEEE Trans Intell Transp Syst 21(11):4615–4625
23.
Kahn G, Villaflor A, Pong V, Abbeel P, Levine S (2017) Uncertainty-aware reinforcement learning for collision avoidance. arXiv preprint arXiv:​1702.​01182
24.
Kim J, Canny J (2017) Interpretable learning for self-driving cars by visualizing causal attention. In Proceedings of the IEEE international conference on computer vision, pp. 2942–2950
25.
Kong Y, Guan Y, Duan J, Li SE, Sun Q, Nie B (2021) Decision-making under on-ramp merge scenarios by distributional soft actor-critic algorithm. arXiv preprint arXiv:​2103.​04535
26.
27.
Lee AX, Nagabandi A, Abbeel P, Levine S (2019) Stochastic latent actor-critic: deep reinforcement learning with a latent variable model. arXiv preprint arXiv:​1907.​00953
28.
29.
Li G, Li SE, Cheng B, Green P (2017) Estimation of driving style in naturalistic highway traffic using maneuver transition probabilities. Transp Res Part C Emerg Technol 74:113–125
30.
Li Shengbo, Li Keqiang, Rajamani Rajesh, Wang Jianqiang (2010) Model predictive multi-objective vehicular adaptive cruise control. IEEE Trans Control Syst Technol 19(3):556–566CrossRef
31.
32.
Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2015) Continuous control with deep reinforcement learning. arXiv preprint arXiv:​1509.​02971
33.
Lu X-Y, Wang J, Li SE, Zheng Y (2014) Multiple-vehicle longitudinal collision mitigation by coordinated brake control. Math Probl Eng 2014
34.
Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529
35.
Mu Y, Li SE, Liu C, Sun Q, Nie B, Cheng B, Peng B (2020) Mixed reinforcement learning with additive stochastic uncertainty. arXiv preprint arXiv:​2003.​00848
36.
Yao Mu, Baiyu Peng, Ziqing Gu, Shengbo Eben Li, Chang Liu, Bingbing Nie, Jianfeng Zheng, and Bo Zhang. Mixed reinforcement learning for efficient policy optimization in stochastic environments. In: 2020 20th international conference on control, automation and systems (ICCAS). IEEE, pp 1212–1219
37.
Peng B, Mu Y, Duan J, Guan Y, Li SE, Chen J (2021) Separated proportional-integral lagrangian for chance constrained reinforcement learning. arXiv preprint arXiv:​2102.​08539
38.
Peng B, Mu Y, Guan Y, Li SE, Yin Y, Chen J (2020) Model-based actor-critic with chance constraint for stochastic system. arXiv preprint arXiv:​2012.​10716 2020
39.
Ren Y, Duan J, Li SE, Guan Y, Sun Q (2020) Improving generalization of reinforcement learning with minimax distributional soft actor-critic. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC). IEEE, pp 1–6
40.
Schulman J, Levine S, Abbeel P, Jordan M, Moritz P (2015) Trust region policy optimization. In: International conference on machine learning, pages 1889–1897
41.
Shengbo LI, Yang G, Lian HOU, Hongbo GAO , Jingliang DUAN , Shuang LIANG , WANG Yu, CHENG Bo, LI Keqiang, REN Wei et al (2019) Key technique of deep neural network and its applications in autonomous driving. J Autom Saf Energy 10(2):119
42.
Sutton RS, Szepesvári C, Geramifard A, Bowling MP (2012) Dyna-style planning with linear function approximation and prioritized sweeping. arXiv preprint arXiv:​1206.​3285
43.
Urmson C, Anhalt J, Bagnell D, Baker C, Bittner R, Clark MN, Dolan J, Duggins D, Galatali T, Geyer C et al (2008) Autonomous driving in urban environments: boss and the urban challenge. J Field Robot 25(8):425–466
44.
Wen L, Duan J, Li SE, Xu S, Peng H (2020) Safe reinforcement learning for autonomous vehicles through parallel constrained policy optimization. In: 2020 IEEE 23rd international conference on intelligent transportation systems (ITSC). IEEE, pp 1–7
45.
Xin L, Kong Y, Li SE, Chen J, Guan Y, Tomizuka M, Cheng B (2021) Enable faster and smoother spatio-temporal trajectory planning for autonomous vehicles in constrained dynamic environment. Proc Inst Mech Eng Part D J Autom Eng 235(4):1101–1112
46.
Yin Y, Li SE, Li K, Yang J, Ma F (2020) Self-learning drift control of automated vehicles beyond handling limit after rear-end collision. Transp Saf Environ 2(2):97–105
47.
Zhang F, Gonzales J, Li SE, Borrelli F, Li K (2018) Drift control for cornering maneuver of autonomous vehicles. Mechatronics 54:167–174
Metadata
Title
Self-learning Decision and Control for Highly Automated Vehicles
Authors
Jianyu Chen
Jingliang Duan
Yang Guan
Qi Sun
Yuming Yin
Shengbo Eben Li
Copyright Year
2023
Book
AI-enabled Technologies for Autonomous and Connected Vehicles

Print ISBN: 978-3-031-06779-2

Electronic ISBN: 978-3-031-06780-8

Copyright Year: 2023

DOI
https://doi.org/10.1007/978-3-031-06780-8_11

Premium Partner

    Image Credits