nach oben

Erschienen in:

2022 | OriginalPaper | Buchkapitel

High-Level Decision-Making Non-player Vehicles

verfasst von : Alessandro Pighetti, Luca Forneris, Luca Lazzaroni, Francesco Bellotti, Alessio Capello, Marianna Cossu, Alessandro De Gloria, Riccardo Berta

Erschienen in: Games and Learning Alliance

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Availability of realistic driver models, also able to represent various driving styles, is key to add traffic in serious games on automotive driving. We propose a new architecture for behavioural planning of vehicles, that decide their motion taking high-level decisions, such as “keep lane”, “overtake” and “go to rightmost lane”. This is similar to a driver’s high-level reasoning and takes into account the availability of ever more sophisticated Advanced Driving Assistance Systems (ADAS) in current vehicles. Compared to a low-level decision making system, our model performs better both in terms of safety and average speed. As a significant advantage, the hierarchical approach allows to reduce the number of training steps, which is critical for ML models, by more than one order of magnitude. The developed agent seems to show a more realistic behaviour. We also showed feasibility of training models able to differentiate their performance in a way similar to the driving styles. We believe that such agents could be profitably employed in state of the art SGs for driving, improving the realism of single NPVs and overall traffic.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel A Virtual Ship Evacuation Serious Game: Assessment of Data and Passenger Training

Nächstes Kapitel Influence of a Mixed Reality Game on Students’ Personal Epistemology. An Empirical Study

Massoud, R., Berta, R., Poslad, S., De Gloria, A., Bellotti, F.: IoT sensing for reality-enhanced serious games, a fuel-efficient drive use case. Sensors. 21, 3559 (2021). https://doi.org/10.3390/s21103559CrossRef

Nousias, S., et al.: Exploiting gamification to improve eco-driving behaviour: the GamECAR approach. Electron. Notes Theor. Comput. Sci. 343, 103–116 (2019). https://doi.org/10.1016/j.entcs.2019.04.013CrossRef

The Good Drive, a serious game for learning to drive - Renault Group. https://www.renaultgroup.com/en/news-on-air/news/the-good-drive-a-serious-game-for-learning-to-drive/. Accessed 22 July 2022

Leurent, E., Mercat, J.: Social Attention for Autonomous Decision-Making in Dense Traffic (2019). https://doi.org/10.48550/arXiv.1911.12250

González, D., Pérez, J., Milanés, V., Nashashibi, F.: A review of motion planning techniques for automated vehicles. IEEE Trans. Intell. Transp. Syst. 17, 1135–1145 (2016). https://doi.org/10.1109/TITS.2015.2498841CrossRef

Francois-Lavet, V., Henderson, P., Islam, R., Bellemare, M.G., Pineau, J.: An introduction to deep reinforcement learning. Found. Trends® Mach. Learn. 11, 219–354 (2018). https://doi.org/10.1561/2200000071

Arulkumaran, K., Deisenroth, M.P., Brundage, M., Bharath, A.A.: A brief survey of deep reinforcement learning. IEEE Signal Process. Mag. 34, 26–38 (2017). https://doi.org/10.1109/MSP.2017.2743240CrossRef

GitHub - eleurent/highway-env: A minimalist environment for decision-making in autonomous driving. https://github.com/eleurent/highway-env. Accessed 11 July 2022

Brockman, G., et al.: OpenAI Gym. (2016). https://doi.org/10.48550/arXiv.1606.01540

10.

Campodonico, G., et al.: Adapting autonomous agents for automotive driving games. In: de Rosa, F., Marfisi Schottman, I., Baalsrud Hauge, J., Bellotti, F., Dondio, P., Romero, M. (eds.) GALA 2021. LNCS, vol. 13134, pp. 101–110. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-92182-8_10CrossRef

11.

Rana, A., Malhi, A.: Building safer autonomous agents by leveraging risky driving behavior knowledge. In: 2021 International Conference on Communications, Computing, Cybersecurity, and Informatics (CCCI), pp. 1–6 (2021). https://doi.org/10.1109/CCCI52664.2021.9583209

12.

Rais, M.S., Boudour, R., Zouaidia, K., Bougueroua, L.: Decision making for autonomous vehicles in highway scenarios using Harmonic SK Deep SARSA. Appl. Intell. (2022). https://doi.org/10.1007/s10489-022-03357-y

13.

Zhang, S., Wu, Y., Ogai, H., Inujima, H., Tateno, S.: Tactical decision-making for autonomous driving using dueling double deep Q network with double attention. IEEE Access. 9, 151983–151992 (2021). https://doi.org/10.1109/ACCESS.2021.3127105CrossRef

14.

SL, UT: Car Driving School Simulator (Android). https://car-driving-school-simulator.en.uptodown.com/android. Accessed 22 July 2022

15.

Gounaridou, A., Siamtanidou, E., Dimoulas, C.: A serious game for mediated education on traffic behavior and safety awareness. Educ. Sci. 11, 127 (2021). https://doi.org/10.3390/educsci11030127CrossRef

16.

Likitweerawong, K., Palee, P.: The virtual reality serious game for learning driving skills before taking practical test. In: 2018 International Conference on Digital Arts Media Technology, ICDAMT (2018). https://doi.org/10.1109/ICDAMT.2018.8376515

17.

Hrimech, H., et al.: The effects of the use of serious game in eco-driving training. Front. ICT. 3 (2016)

18.

Massoud, R., Poslad, S., Bellotti, F., Berta, R., Mehran, K., Gloria, A.D.: A fuzzy logic module to estimate a driver’s fuel consumption for reality-enhanced serious games. Int. J. Serious Games 5, 45–62 (2018). https://doi.org/10.17083/ijsg.v5i4.266

19.

Lazzaroni, L., Mazzara, A., Bellotti, F., De Gloria, A., Berta, R.: Employing an IoT framework as a generic serious games analytics engine. In: Marfisi-Schottman, I., Bellotti, F., Hamon, L., Klemke, R. (eds.) GALA 2020. LNCS, vol. 12517, pp. 79–88. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-63464-3_8CrossRef

20.

Westera, W., et al.: Artificial intelligence moving serious gaming: presenting reusable game AI components. Educ. Inf. Technol. 25(1), 351–380 (2019). https://doi.org/10.1007/s10639-019-09968-2CrossRef

21.

Tomlinson, S.L., Melder, N.: An architecture overview for AI in racing games. (2015). https://doi.org/10.1201/b16725-44

22.

Perot, E., Jaritz, M., Toromanoff, M., De Charette, R.: End-to-end driving in a realistic racing game with deep reinforcement learning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 474–475 (2017). https://doi.org/10.1109/CVPRW.2017.64

23.

Fakhry, A.: Applying a Deep Q Network for OpenAI’s Car Racing Game. https://towardsdatascience.com/applying-a-deep-q-network-for-openais-car-racing-game-a642daf58fc9. Accessed 22 July 2022

24.

Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015). https://doi.org/10.1038/nature14236CrossRef

25.

Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992). https://doi.org/10.1007/BF00992698CrossRefMATH

26.

Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal Policy Optimization Algorithms (2017). https://doi.org/10.48550/arXiv.1707.06347

27.

Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: Advances in Neural Information Processing Systems. MIT Press (1999)

28.

Wang, T., et al.: Benchmarking Model-Based Reinforcement Learning (2019). https://doi.org/10.48550/arXiv.1907.02057

29.

Raffin, A., Hill, A., Gleave, A., Kanervisto, A., Ernestus, M., Dormann, N.: Stable-Baselines3: reliable reinforcement learning implementations. J. Mach. Learn. Res. 22, 1–8 (2021)MATH

30.

Mirchevska, B., Pek, C., Werling, M., Althoff, M., Boedecker, J.: High-level decision making for safe and reasonable autonomous lane changing using reinforcement learning. In: 2018 21st International Conference on Intelligent Transportation Systems (ITSC), pp. 2156–2162 (2018). https://doi.org/10.1109/ITSC.2018.8569448

31.

PPO—Stable Baselines3 1.6.1a0 documentation. https://stable-baselines3.readthedocs.io/en/master/modules/ppo.html. Accessed 23 July 2022

32.

Abadi, M., et al.: TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems (2016). https://doi.org/10.48550/arXiv.1603.04467

33.

Kirkpatrick, J., et al.: Overcoming catastrophic forgetting in neural networks. Proc. Natl. Acad. Sci. 114, 3521–3526 (2017). https://doi.org/10.1073/pnas.1611835114MathSciNetCrossRefMATH

Titel: High-Level Decision-Making Non-player Vehicles
verfasst von: Alessandro Pighetti
Luca Forneris
Luca Lazzaroni
Francesco Bellotti
Alessio Capello
Marianna Cossu
Alessandro De Gloria
Riccardo Berta
Verlag: Springer International Publishing
Buch: Games and Learning Alliance
Print ISBN: 978-3-031-22123-1

Electronic ISBN: 978-3-031-22124-8

Copyright-Jahr: 2022
DOI: https://doi.org/10.1007/978-3-031-22124-8_22

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner