Top

Published in:

2024 | OriginalPaper | Chapter

Non-linear Reward Deep Q Networks for Smooth Action in a Car Game

Authors : Mohammad Iqbal, Achmad Afandy, Nurul Hidayat

Published in: Applied and Computational Mathematics

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

We formulate non-linear reward functions on deep Q networks in a car racing game by observing the environment (simulator). We aim to control the car movement (action) more smoothly in the game simulator than in the original. Existing studies about deep reinforcement learning maintained either discrete or non-linear reward functions without considering the environment domain, which may lead to illogical car movements. For instance, the car is blocked by three other cars, yet the game still continues by jumping to one of them. To overcome the issues, we define a non-linear reward function to compute the penalty game score based on the distance between the car and the one in front of it. From the game simulator, we surprisingly enjoy the results from the proposed reward function as the car drives more accurately and smoothly than the SOTA models, even at the start of the game point, by showing the smallest number of crashes and no zigzaggy agent movement when the obstacles are far from it.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter A Trustworthiness Scoring Model for Reviews Using a Graph Structure

next chapter College Student Activity Recognition from Smartwatch Dataset

https://www.deepmind.com/research/highlighted-research/alphago.

The game simulator code can be downloaded from https://github.com/MajidMoghadam2006/deepcars-reinforcement-learning.

Busoniu, L., Babuska, R., De Schutter, B., Ernst, D.: Reinforcement Learning and Dynamic Programming Using Function Approximators. CRC Press (2017)

Dai, B., Shaw, A., Li, L., Xiao, L., He, N., Liu, Z., Chen, J., Song, L.: SBEED: Convergent reinforcement learning with nonlinear function approximation. In: Dy, J., Krause, A. (eds.) Proceedings of the 35th International Conference on Machine Learning. Proceedings of Machine Learning Research, vol. 80, pp. 1125–1134. PMLR, 10–15 Jul 2018. https://proceedings.mlr.press/v80/dai18c.html

Dlugosz, Z., Dlugosz, R.: Nonlinear activation functions for artificial neural networks realized in hardware. In: 2018 25th International Conference Mixed Design of Integrated Circuits and System (MIXDES), pp. 381–384 (2018)

Grigorescu, S., Trasnea, B., Cocias, T., Macesanu, G.: A survey of deep learning techniques for autonomous driving. J. Field Robotics 37(3), 362–386 (2020). https://onlinelibrary.wiley.com/doi/abs/10.1002/rob.21918

Haider, A., Hawe, G., Wang, H., Scotney, B.: Gaussian based non-linear function approximation for reinforcement learning. SN Comput. Sci. 2(223) (2021). https://link.springer.com/article/10.1007/s42979-021-00642-4

Hasselt, H.V., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. In: Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence (AAAI’16), pp. 2094–2100. AAAI Press (2016)

Karalakou, A., Troullinos, D., Chalkiadakis, G., Papageorgiou, M.: Deep reinforcement learning reward function design for autonomous driving in lane-free traffic. Systems 11(3) (2023). https://www.mdpi.com/2079-8954/11/3/134

Kiran, B.R., Sobh, I., Talpaert, V., Mannion, P., Sallab, A.A.A., Yogamani, S., Pérez, P.: Deep reinforcement learning for autonomous driving: a survey. IEEE Trans. Intell. Transp. Syst. 23(6), 4909–4926 (2022)CrossRef

Lin, C.C., Deng, D.J., Chih, Y.L., Chiu, H.T.: Smart manufacturing scheduling with edge computing using multiclass deep q network. IEEE Trans. Industr. Inf. 15(7), 4276–4284 (2019)CrossRef

10.

Mnih, V., Kavukcuoglu, K., Silver, D., Graves, A., Antonoglou, I., Wierstra, D., Riedmiller, M.: Playing atari with deep reinforcement learning (2013)

11.

Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M., Fidjeland, A.K., Ostrovski, G., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015)

12.

Moghadam, M., Elkaim, G.H.: A hierarchical architecture for sequential decision-making in autonomous driving using deep reinforcement learning (2019). Preprint at arXiv:1906.08464

13.

Watkins, C.J., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

Title: Non-linear Reward Deep Q Networks for Smooth Action in a Car Game
Authors: Mohammad Iqbal
Achmad Afandy
Nurul Hidayat
Publisher: Springer Nature Singapore
Book: Applied and Computational Mathematics
Print ISBN: 978-981-9721-35-1

Electronic ISBN: 978-981-9721-36-8

Copyright Year: 2024
DOI: https://doi.org/10.1007/978-981-97-2136-8_19

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"