nach oben

International Journal of Intelligent Transportation Systems Research

Erschienen in:

07.10.2024

CM3-VSL: Cooperative Multi-goal Multi-stage Multi-agent VSL Traffic Control

verfasst von: Asmae Rhanizar, Zineb El Akkaoui

Erschienen in: International Journal of Intelligent Transportation Systems Research | Ausgabe 3/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Variable speed limit (VSL) Systems play a crucial role in proactively optimizing traffic control. As a matter of fact, many countries have deployed VSL Systems to improve road safety and resolve traffic breakdown. Most of smart VSL strategies are deployed to optimize traffic flow within a single road segment only, while real-world scenarios often involve complex bottleneck situations arising from multiple ramps. In response, we introduce a novel Cooperative Multi-goal Multi-stage Multi-agent VSL (CM3-VSL) framework where a diverse set of VSL agents collaboratively work towards both individualized local goals and shared global objectives, addressing the complexities of real-world traffic scenarios. The VSL agents are trained using micro-simulations on a real-world Moroccan highway network. Employing a cooperative strategy, each VSL agent pursues both individual and collective goals. Evaluation against a baseline no-VSL scenario and a single-agent multi-objective Reinforcement Learning VSL demonstrates that CM3-VSL achieves superior performance, contributing to advancements in intelligent traffic control systems.

Vorheriger Artikel Edge Cloud Assisted Quantum LSTM-based Framework for Road Traffic Monitoring

Nächster Artikel Advanced Modeling and Interpretation for Accurate Intersection Traffic Time Prediction

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information.

Order your 30-days-trial for free and without any commitment.

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

ATZelektronik

Die Fachzeitschrift ATZelektronik bietet für Entwickler und Entscheider in der Automobil- und Zulieferindustrie qualitativ hochwertige und fundierte Informationen aus dem gesamten Spektrum der Pkw- und Nutzfahrzeug-Elektronik.

Lassen Sie sich jetzt unverbindlich 2 kostenlose Ausgabe zusenden.

Jetzt informieren

Lee, C., Hellinga, B., Saccomanno, F.: Evaluation of variable speed limits to improve traffic safety. Transp. Res. C: Emerg. Technol. 14(3), 213–228 (2006). https://doi.org/10.1016/j.trc.2006.06.002CrossRef

Pu, Z., Li, Z., Jiang, Y., Wang, Y.: Full bayesian before-after analysis of safety effects of variable speed limit system. IEEE Trans. Intell. Transp. Syst. 22(2), 964–976 (2021). https://doi.org/10.1109/TITS.2019.2961699CrossRef

Zhu, F., Ukkusuri, S.V.: Accounting for dynamic speed limit control in a stochastic traffic environment: A reinforcement learning approach. Transp. Res. C: Emerg. Technol. 41, 30–47 (2014). https://doi.org/10.1016/j.trc.2014.01.014CrossRef

Asmae, R., Zineb, E.A.: Learning-based Variable speed limit control strategies: RL, DRL and MARL. MAIDSS’24: Proceedings of the 2024 International Conference on Modern Artificial Intelligence and Data Science Systems (2024). https://doi.org/10.1007/978-3-031-65038-3_45

Kušić, K., Ivanjko, E., Gregurić, M., Miletić, M.: An overview of reinforcement learning methods for variable speed limit control. Appl. Sci. 10(14), 4917 (2020). https://doi.org/10.3390/app10144917CrossRef

Li, Z., Xu, C., Guo, Y., Liu, P., Pu, Z.: Reinforcement learning-based variable speed limits control to reduce crash risks near traffic oscillations on freeways. IEEE Intell. Transp. Syst. Mag. 13(4), 64–70 (2021). https://doi.org/10.1109/MITS.2019.2907631CrossRef

Adler, J.L., Satapathy, G., Manikonda, V., Bowles, B., Blue, V.J.: A multi-agent approach to cooperative traffic management and route guidance. Transp. Res. B Methodol. 39(4), 297–318 (2005). https://doi.org/10.1016/j.trb.2004.03.005CrossRef

Hamidi, H., Kamankesh, A.: An approach to intelligent traffic management system using a multi-agent system. Int. J. Intell. Transp. Syst. Res. 16(2), 112–124 (2018). https://doi.org/10.1007/s13177-017-0142-6CrossRef

Hirankitti, V., Krohkaew, J., Hogger, C.: A multi-agent approach for intelligent traffic-light control (2007)

10.

Ikidid, A., Abdelaziz, E.F., Sadgal, M.: Multi-agent and fuzzy inference-based framework for traffic light optimization. Int. J. Interact. Multimed. Artif. Intell. 8(2), 88 (2023). https://doi.org/10.9781/ijimai.2021.12.002CrossRef

11.

Cetin, N., Nagel, K., Raney, B., Voellmy, A.: Large-scale multi-agent transportation simulations. Comput. Phys. Commun. 147(1–2), 559–564 (2002). https://doi.org/10.1016/S0010-4655(02)00353-3CrossRef

12.

Tao, C., Huang, S.: An extensible multi-agent based traffic simulation system. In: 2009 International Conference on Measuring Technology and Mechatronics Automation, pp. 713–716. IEEE, Zhangjiajie, Hunan, China (2009). https://doi.org/10.1109/ICMTMA.2009.42 . http://ieeexplore.ieee.org/document/5203302/ Accessed 04 Dec 2023

13.

Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: An Overview. In: Kacprzyk, J., Srinivasan, D., Jain, L.C. (eds.) Innovations in Multi-Agent Systems and Applications - 1 vol. 310, pp. 183–221. Springer, Berlin, Heidelberg (2010)

14.

Nguyen, T.T., Nguyen, N.D., Nahavandi, S.: Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications. IEEE Trans. Cybern. 50(9), 3826–3839 (2020). https://doi.org/10.1109/TCYB.2020.2977374CrossRef

15.

Perrusquía, A., Yu, W., Li, X.: Multi-agent reinforcement learning for redundant robot control in task-space. Int. J. Mach. Learn. Cybern. 12(1), 231–241 (2021). https://doi.org/10.1007/s13042-020-01167-7CrossRef

16.

OroojlooyJadid, A., Hajinezhad, D.: A review of cooperative multi-agent deep reinforcement learning. arXiv (2021)

17.

Sharma, P.K., Zaroukian, E.G., Fernandez, R., Basak, A., Asher, D.E.: Survey of recent multi-agent reinforcement learning algorithms utilizing centralized training. In: Pham, T., Solomon, L., Hohil, M.E. (eds.) Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III, p. 84. SPIE, Online Only, United States (2021). https://doi.org/10.1117/12.2585808

18.

Wang, C., Zhang, J., Xu, L., Li, L., Ran, B.: A new solution for freeway congestion: Cooperative speed limit control using distributed reinforcement learning. IEEE Access 7, 41947–41957 (2019). https://doi.org/10.1109/ACCESS.2019.2904619CrossRef

19.

Kušić, K., Ivanjko, E., Vrbanić, F., Gregurić, M., Dusparic, I.: Spatial-temporal traffic flow control on motorways using distributed multi-agent reinforcement learning. Mathematics 9(23), 3081 (2021). https://doi.org/10.3390/math9233081CrossRef

20.

Zheng, S., Li, M., Ke, Z., Li, Z.: Coordinated variable speed limit control for consecutive bottlenecks on freeways using multiagent reinforcement learning. J. Adv. Transp. 2023, 1–19 (2023). https://doi.org/10.1155/2023/4419907CrossRef

21.

Fang, X., Péter, T., Tettamanti, T.: Variable speed limit control for the motorway-urban merging bottlenecks using multi-agent reinforcement learning. Sustainability 15(14), 11464 (2023). https://doi.org/10.3390/su151411464CrossRef

22.

Zhang, Y., Quinones-Grueiro, M., Zhang, Z., Wang, Y., Barbour, W., Biswas, G., Work, D.: MARVEL: Multi-agent reinforcement-learning for large-scale variable speed limits. arXiv (2023)

23.

Yang, J., Nakhaei, A., Isele, D., Fujimura, K., Zha, H.: CM3: Cooperative multi-goal multi-stage multi-agent reinforcement learning. arXiv (2020)

24.

Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Machine learning proceedings 1994, pp. 157–163. Elsevier (1994). https://doi.org/10.1016/B978-1-55860-335-6.50027-1. https://linkinghub.elsevier.com/retrieve/pii/B9781558603356500271. Accessed 29 Sept 2023

25.

Mushtaq, A., Haq, I.U., Sarwar, M.A., Khan, A., Khalil, W., Mughal, M.A.: Multi-agent reinforcement learning for traffic flow management of autonomous vehicles. Sensors 23(5), 2373 (2023). https://doi.org/10.3390/s23052373. Accessed 29 Sept 2023

26.

Yang, Y., Wang, J.: An overview of multi-agent reinforcement learning from game theoretical perspective. arXiv. arXiv:2011.00583 [cs] (2021). http://arxiv.org/abs/2011.00583. Accessed 13 Nov 2023

27.

Tan, M.: Multi-agent reinforcement learning: Independent versus cooperative agents. In: International conference on machine learning (1997). https://api.semanticscholar.org/CorpusID:266591061

28.

Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. arXiv (2020)

29.

Ryu, H., Shin, H., Park, J.: Multi-agent actor-critic with generative cooperative policy network (2018). https://doi.org/10.48550/ARXIV.1810.09206

30.

Mao, H., Zhang, Z., Xiao, Z., Gong, Z.: Modelling the dynamic joint policy of teammates with attention multi-agent DDPG (2018). https://doi.org/10.48550/ARXIV.1811.07029 . Publisher: arXiv Version Number: 1

31.

Wang, R.E., Everett, M., How, J.P.: R-MADDPG for partially observable environments and limited communication (2020). https://doi.org/10.48550/ARXIV.2002.06684. Publisher: arXiv Version Number: 2

32.

Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., Whiteson, S.: Counterfactual multi-agent policy gradients. arXiv (2017)

33.

Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W.M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J.Z., Tuyls, K., Graepel, T.: Value-decomposition networks for cooperative multi-agent learning (2017). https://doi.org/10.48550/ARXIV.1706.05296

34.

Rashid, T., Samvelyan, M., Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning (2018). https://doi.org/10.48550/ARXIV.1803.11485. Publisher: arXiv Version Number: 2

35.

Macua, S.V., Tukiainen, A., Hernández, D.G.-O., Baldazo, D., Cote, E.M., Zazo, S.: Diff-DAC: Distributed actor-critic for average multitask deep reinforcement learning (2017). https://doi.org/10.48550/ARXIV.1710.10363

36.

Zhang, K., Yang, Z., Liu, H., Zhang, T., Başar, T.: Fully decentralized multi-agent reinforcement learning with networked agents (2018). https://doi.org/10.48550/ARXIV.1802.08757

37.

Foerster, J.N., Assael, Y.M., Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning (2016). https://doi.org/10.48550/ARXIV.1605.06676

38.

Jorge, E., Kågebäck, M., Johansson, F.D., Gustavsson, E.: Learning to Play Guess Who? and inventing a grounded language as a consequence (2016). https://doi.org/10.48550/ARXIV.1611.03218

39.

Le, N.-T.-T.: Multi-agent reinforcement learning for traffic congestion on one-way multi-lane highways. J. Inf. Telecommun. 7(3), 255–269 (2023). https://doi.org/10.1080/24751839.2023.2182174CrossRef

40.

Calvo, J.A., Dusparic, I.: Heterogeneous multi-agent deep reinforcement learning for traffic lights control (2018)

41.

Wang, X., Ke, L., Qiao, Z., Chai, X.: Large-scale traffic signal control using a novel multiagent reinforcement learning. IEEE Trans. Cybern. 51(1), 174–187 (2021). https://doi.org/10.1109/TCYB.2020.3015811CrossRef

42.

Puterman, M.L.: Markov decision processes: Discrete stochastic dynamic programming, 1st edn. Wiley series in probability and statistics. Wiley, (1994). https://doi.org/10.1002/9780470316887

43.

Asmae, R., Zineb, E.A.: Multi-objective deep reinforcement learning for variable speed limit control. In: Computing Machinery, A. (ed.) ICMLC ’24: Proceedings of the 2024 16th International Conference on Machine Learning and Computing, New York (2024). https://doi.org/10.1145/3651671.3651719

44.

Mahmud, S.M.S., Ferreira, L., Hoque, M.S., Tavassoli, A.: Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs. IATSS Research 41(4), 153–163 (2017). https://doi.org/10.1016/j.iatssr.2017.02.001CrossRef

Titel: CM3-VSL: Cooperative Multi-goal Multi-stage Multi-agent VSL Traffic Control
verfasst von: Asmae Rhanizar
Zineb El Akkaoui
Publikationsdatum: 07.10.2024
Verlag: Springer US
Erschienen in: International Journal of Intelligent Transportation Systems Research / Ausgabe 3/2024
Print ISSN: 1348-8503
Elektronische ISSN: 1868-8659
DOI: https://doi.org/10.1007/s13177-024-00426-z

DA-RNN-Based Bus Arrival Time Prediction Model

Multiclass Traffic Assignment Model Considering Heterogeneous Stochastic Headways of Autonomous Vehicles and Human-Driven Vehicles

An Effective Traffic Management Framework for Congestion Prediction and Re-Routing Using Hybridized Techniques

Predicting Curb Side Parking Availability for Commercial Vehicle Loading Zones

Minimizing Intersection Delays: A Novel Fuzzy Logic-Based Architecture for Traffic Signal Control

Passenger-oriented Distributed Traffic Signal Control Using Dynamic Programming with Vehicle Queue Spillback and Waiting Time Constraints

Premium Partner

Marktübersichten

Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.

Zur Marktübersicht

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

ATZelectronics worldwide

Springer Professional "Technik"

ATZelektronik

DA-RNN-Based Bus Arrival Time Prediction Model

Multiclass Traffic Assignment Model Considering Heterogeneous Stochastic Headways of Autonomous Vehicles and Human-Driven Vehicles

An Effective Traffic Management Framework for Congestion Prediction and Re-Routing Using Hybridized Techniques

Predicting Curb Side Parking Availability for Commercial Vehicle Loading Zones

Minimizing Intersection Delays: A Novel Fuzzy Logic-Based Architecture for Traffic Signal Control

Passenger-oriented Distributed Traffic Signal Control Using Dynamic Programming with Vehicle Queue Spillback and Waiting Time Constraints

Marktübersichten