Top

International Journal of Intelligent Transportation Systems Research

Published in:

07-10-2024

CM3-VSL: Cooperative Multi-goal Multi-stage Multi-agent VSL Traffic Control

Authors: Asmae Rhanizar, Zineb El Akkaoui

Published in: International Journal of Intelligent Transportation Systems Research | Issue 3/2024

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

The article introduces the CM3-VSL framework, a cooperative multi-goal multi-stage multi-agent system for Variable Speed Limit (VSL) traffic control. It leverages deep neural networks and reinforcement learning to optimize traffic flow and safety. The framework allows VSL agents to pursue individual local goals while collaborating to achieve global objectives such as safety, mobility, and environmental sustainability. The study validates the CM3-VSL framework through micro-simulations on a Moroccan highway network, demonstrating its effectiveness in enhancing traffic flow and safety compared to traditional VSL strategies. The article also discusses the challenges and advantages of using multi-agent reinforcement learning in traffic control systems, making it a valuable resource for professionals in transportation engineering and traffic management.

AI Generated

This summary of the content was generated with the help of AI.

previous article Edge Cloud Assisted Quantum LSTM-based Framework for Road Traffic Monitoring

next article Advanced Modeling and Interpretation for Accurate Intersection Traffic Time Prediction

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information.

Order your 30-days-trial for free and without any commitment.

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

ATZelektronik

Die Fachzeitschrift ATZelektronik bietet für Entwickler und Entscheider in der Automobil- und Zulieferindustrie qualitativ hochwertige und fundierte Informationen aus dem gesamten Spektrum der Pkw- und Nutzfahrzeug-Elektronik.

Lassen Sie sich jetzt unverbindlich 2 kostenlose Ausgabe zusenden.

inform now

Lee, C., Hellinga, B., Saccomanno, F.: Evaluation of variable speed limits to improve traffic safety. Transp. Res. C: Emerg. Technol. 14(3), 213–228 (2006). https://doi.org/10.1016/j.trc.2006.06.002CrossRef

Pu, Z., Li, Z., Jiang, Y., Wang, Y.: Full bayesian before-after analysis of safety effects of variable speed limit system. IEEE Trans. Intell. Transp. Syst. 22(2), 964–976 (2021). https://doi.org/10.1109/TITS.2019.2961699CrossRef

Zhu, F., Ukkusuri, S.V.: Accounting for dynamic speed limit control in a stochastic traffic environment: A reinforcement learning approach. Transp. Res. C: Emerg. Technol. 41, 30–47 (2014). https://doi.org/10.1016/j.trc.2014.01.014CrossRef

Asmae, R., Zineb, E.A.: Learning-based Variable speed limit control strategies: RL, DRL and MARL. MAIDSS’24: Proceedings of the 2024 International Conference on Modern Artificial Intelligence and Data Science Systems (2024). https://doi.org/10.1007/978-3-031-65038-3_45

Kušić, K., Ivanjko, E., Gregurić, M., Miletić, M.: An overview of reinforcement learning methods for variable speed limit control. Appl. Sci. 10(14), 4917 (2020). https://doi.org/10.3390/app10144917CrossRef

Li, Z., Xu, C., Guo, Y., Liu, P., Pu, Z.: Reinforcement learning-based variable speed limits control to reduce crash risks near traffic oscillations on freeways. IEEE Intell. Transp. Syst. Mag. 13(4), 64–70 (2021). https://doi.org/10.1109/MITS.2019.2907631CrossRef

Adler, J.L., Satapathy, G., Manikonda, V., Bowles, B., Blue, V.J.: A multi-agent approach to cooperative traffic management and route guidance. Transp. Res. B Methodol. 39(4), 297–318 (2005). https://doi.org/10.1016/j.trb.2004.03.005CrossRef

Hamidi, H., Kamankesh, A.: An approach to intelligent traffic management system using a multi-agent system. Int. J. Intell. Transp. Syst. Res. 16(2), 112–124 (2018). https://doi.org/10.1007/s13177-017-0142-6CrossRef

Hirankitti, V., Krohkaew, J., Hogger, C.: A multi-agent approach for intelligent traffic-light control (2007)

10.

Ikidid, A., Abdelaziz, E.F., Sadgal, M.: Multi-agent and fuzzy inference-based framework for traffic light optimization. Int. J. Interact. Multimed. Artif. Intell. 8(2), 88 (2023). https://doi.org/10.9781/ijimai.2021.12.002CrossRef

11.

Cetin, N., Nagel, K., Raney, B., Voellmy, A.: Large-scale multi-agent transportation simulations. Comput. Phys. Commun. 147(1–2), 559–564 (2002). https://doi.org/10.1016/S0010-4655(02)00353-3CrossRef

12.

Tao, C., Huang, S.: An extensible multi-agent based traffic simulation system. In: 2009 International Conference on Measuring Technology and Mechatronics Automation, pp. 713–716. IEEE, Zhangjiajie, Hunan, China (2009). https://doi.org/10.1109/ICMTMA.2009.42 . http://ieeexplore.ieee.org/document/5203302/ Accessed 04 Dec 2023

13.

Buşoniu, L., Babuška, R., De Schutter, B.: Multi-agent reinforcement learning: An Overview. In: Kacprzyk, J., Srinivasan, D., Jain, L.C. (eds.) Innovations in Multi-Agent Systems and Applications - 1 vol. 310, pp. 183–221. Springer, Berlin, Heidelberg (2010)

14.

Nguyen, T.T., Nguyen, N.D., Nahavandi, S.: Deep reinforcement learning for multiagent systems: A review of challenges, solutions, and applications. IEEE Trans. Cybern. 50(9), 3826–3839 (2020). https://doi.org/10.1109/TCYB.2020.2977374CrossRef

15.

Perrusquía, A., Yu, W., Li, X.: Multi-agent reinforcement learning for redundant robot control in task-space. Int. J. Mach. Learn. Cybern. 12(1), 231–241 (2021). https://doi.org/10.1007/s13042-020-01167-7CrossRef

16.

OroojlooyJadid, A., Hajinezhad, D.: A review of cooperative multi-agent deep reinforcement learning. arXiv (2021)

17.

Sharma, P.K., Zaroukian, E.G., Fernandez, R., Basak, A., Asher, D.E.: Survey of recent multi-agent reinforcement learning algorithms utilizing centralized training. In: Pham, T., Solomon, L., Hohil, M.E. (eds.) Artificial Intelligence and Machine Learning for Multi-Domain Operations Applications III, p. 84. SPIE, Online Only, United States (2021). https://doi.org/10.1117/12.2585808

18.

Wang, C., Zhang, J., Xu, L., Li, L., Ran, B.: A new solution for freeway congestion: Cooperative speed limit control using distributed reinforcement learning. IEEE Access 7, 41947–41957 (2019). https://doi.org/10.1109/ACCESS.2019.2904619CrossRef

19.

Kušić, K., Ivanjko, E., Vrbanić, F., Gregurić, M., Dusparic, I.: Spatial-temporal traffic flow control on motorways using distributed multi-agent reinforcement learning. Mathematics 9(23), 3081 (2021). https://doi.org/10.3390/math9233081CrossRef

20.

Zheng, S., Li, M., Ke, Z., Li, Z.: Coordinated variable speed limit control for consecutive bottlenecks on freeways using multiagent reinforcement learning. J. Adv. Transp. 2023, 1–19 (2023). https://doi.org/10.1155/2023/4419907CrossRef

21.

Fang, X., Péter, T., Tettamanti, T.: Variable speed limit control for the motorway-urban merging bottlenecks using multi-agent reinforcement learning. Sustainability 15(14), 11464 (2023). https://doi.org/10.3390/su151411464CrossRef

22.

Zhang, Y., Quinones-Grueiro, M., Zhang, Z., Wang, Y., Barbour, W., Biswas, G., Work, D.: MARVEL: Multi-agent reinforcement-learning for large-scale variable speed limits. arXiv (2023)

23.

Yang, J., Nakhaei, A., Isele, D., Fujimura, K., Zha, H.: CM3: Cooperative multi-goal multi-stage multi-agent reinforcement learning. arXiv (2020)

24.

Littman, M.L.: Markov games as a framework for multi-agent reinforcement learning. In: Machine learning proceedings 1994, pp. 157–163. Elsevier (1994). https://doi.org/10.1016/B978-1-55860-335-6.50027-1. https://linkinghub.elsevier.com/retrieve/pii/B9781558603356500271. Accessed 29 Sept 2023

25.

Mushtaq, A., Haq, I.U., Sarwar, M.A., Khan, A., Khalil, W., Mughal, M.A.: Multi-agent reinforcement learning for traffic flow management of autonomous vehicles. Sensors 23(5), 2373 (2023). https://doi.org/10.3390/s23052373. Accessed 29 Sept 2023

26.

Yang, Y., Wang, J.: An overview of multi-agent reinforcement learning from game theoretical perspective. arXiv. arXiv:2011.00583 [cs] (2021). http://arxiv.org/abs/2011.00583. Accessed 13 Nov 2023

27.

Tan, M.: Multi-agent reinforcement learning: Independent versus cooperative agents. In: International conference on machine learning (1997). https://api.semanticscholar.org/CorpusID:266591061

28.

Lowe, R., Wu, Y., Tamar, A., Harb, J., Abbeel, P., Mordatch, I.: Multi-agent actor-critic for mixed cooperative-competitive environments. arXiv (2020)

29.

Ryu, H., Shin, H., Park, J.: Multi-agent actor-critic with generative cooperative policy network (2018). https://doi.org/10.48550/ARXIV.1810.09206

30.

Mao, H., Zhang, Z., Xiao, Z., Gong, Z.: Modelling the dynamic joint policy of teammates with attention multi-agent DDPG (2018). https://doi.org/10.48550/ARXIV.1811.07029 . Publisher: arXiv Version Number: 1

31.

Wang, R.E., Everett, M., How, J.P.: R-MADDPG for partially observable environments and limited communication (2020). https://doi.org/10.48550/ARXIV.2002.06684. Publisher: arXiv Version Number: 2

32.

Foerster, J., Farquhar, G., Afouras, T., Nardelli, N., Whiteson, S.: Counterfactual multi-agent policy gradients. arXiv (2017)

33.

Sunehag, P., Lever, G., Gruslys, A., Czarnecki, W.M., Zambaldi, V., Jaderberg, M., Lanctot, M., Sonnerat, N., Leibo, J.Z., Tuyls, K., Graepel, T.: Value-decomposition networks for cooperative multi-agent learning (2017). https://doi.org/10.48550/ARXIV.1706.05296

34.

Rashid, T., Samvelyan, M., Witt, C.S., Farquhar, G., Foerster, J., Whiteson, S.: QMIX: Monotonic value function factorisation for deep multi-agent reinforcement learning (2018). https://doi.org/10.48550/ARXIV.1803.11485. Publisher: arXiv Version Number: 2

35.

Macua, S.V., Tukiainen, A., Hernández, D.G.-O., Baldazo, D., Cote, E.M., Zazo, S.: Diff-DAC: Distributed actor-critic for average multitask deep reinforcement learning (2017). https://doi.org/10.48550/ARXIV.1710.10363

36.

Zhang, K., Yang, Z., Liu, H., Zhang, T., Başar, T.: Fully decentralized multi-agent reinforcement learning with networked agents (2018). https://doi.org/10.48550/ARXIV.1802.08757

37.

Foerster, J.N., Assael, Y.M., Freitas, N., Whiteson, S.: Learning to communicate with deep multi-agent reinforcement learning (2016). https://doi.org/10.48550/ARXIV.1605.06676

38.

Jorge, E., Kågebäck, M., Johansson, F.D., Gustavsson, E.: Learning to Play Guess Who? and inventing a grounded language as a consequence (2016). https://doi.org/10.48550/ARXIV.1611.03218

39.

Le, N.-T.-T.: Multi-agent reinforcement learning for traffic congestion on one-way multi-lane highways. J. Inf. Telecommun. 7(3), 255–269 (2023). https://doi.org/10.1080/24751839.2023.2182174CrossRef

40.

Calvo, J.A., Dusparic, I.: Heterogeneous multi-agent deep reinforcement learning for traffic lights control (2018)

41.

Wang, X., Ke, L., Qiao, Z., Chai, X.: Large-scale traffic signal control using a novel multiagent reinforcement learning. IEEE Trans. Cybern. 51(1), 174–187 (2021). https://doi.org/10.1109/TCYB.2020.3015811CrossRef

42.

Puterman, M.L.: Markov decision processes: Discrete stochastic dynamic programming, 1st edn. Wiley series in probability and statistics. Wiley, (1994). https://doi.org/10.1002/9780470316887

43.

Asmae, R., Zineb, E.A.: Multi-objective deep reinforcement learning for variable speed limit control. In: Computing Machinery, A. (ed.) ICMLC ’24: Proceedings of the 2024 16th International Conference on Machine Learning and Computing, New York (2024). https://doi.org/10.1145/3651671.3651719

44.

Mahmud, S.M.S., Ferreira, L., Hoque, M.S., Tavassoli, A.: Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs. IATSS Research 41(4), 153–163 (2017). https://doi.org/10.1016/j.iatssr.2017.02.001CrossRef

Title: CM3-VSL: Cooperative Multi-goal Multi-stage Multi-agent VSL Traffic Control
Authors: Asmae Rhanizar
Zineb El Akkaoui
Publication date: 07-10-2024
Publisher: Springer US
Published in: International Journal of Intelligent Transportation Systems Research / Issue 3/2024
Print ISSN: 1348-8503
Electronic ISSN: 1868-8659
DOI: https://doi.org/10.1007/s13177-024-00426-z

A Two-Stage Behavioral Model Considering Vehicle Motion Fluctuations for Decision-Making During Lane Changes in Diverging Areas

An Effective Traffic Management Framework for Congestion Prediction and Re-Routing Using Hybridized Techniques

Passenger-oriented Distributed Traffic Signal Control Using Dynamic Programming with Vehicle Queue Spillback and Waiting Time Constraints

Edge Cloud Assisted Quantum LSTM-based Framework for Road Traffic Monitoring

Multi-Model Traffic Forecasting in Smart Cities using Graph Neural Networks and Transformer-based Multi-Source Visual Fusion for Intelligent Transportation Management

Springer Professional

CM3-VSL: Cooperative Multi-goal Multi-stage Multi-agent VSL Traffic Control

Abstract

Abstract

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

ATZelectronics worldwide

Springer Professional "Technik"

ATZelektronik

Other articles of this Issue 3/2024

A Two-Stage Behavioral Model Considering Vehicle Motion Fluctuations for Decision-Making During Lane Changes in Diverging Areas

An Effective Traffic Management Framework for Congestion Prediction and Re-Routing Using Hybridized Techniques

Passenger-oriented Distributed Traffic Signal Control Using Dynamic Programming with Vehicle Queue Spillback and Waiting Time Constraints

Edge Cloud Assisted Quantum LSTM-based Framework for Road Traffic Monitoring

Multi-Model Traffic Forecasting in Smart Cities using Graph Neural Networks and Transformer-based Multi-Source Visual Fusion for Intelligent Transportation Management

DA-RNN-Based Bus Arrival Time Prediction Model

Premium Partners

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

ATZelectronics worldwide

Springer Professional "Technik"

ATZelektronik

A Two-Stage Behavioral Model Considering Vehicle Motion Fluctuations for Decision-Making During Lane Changes in Diverging Areas

An Effective Traffic Management Framework for Congestion Prediction and Re-Routing Using Hybridized Techniques

Passenger-oriented Distributed Traffic Signal Control Using Dynamic Programming with Vehicle Queue Spillback and Waiting Time Constraints

Edge Cloud Assisted Quantum LSTM-based Framework for Road Traffic Monitoring

Multi-Model Traffic Forecasting in Smart Cities using Graph Neural Networks and Transformer-based Multi-Source Visual Fusion for Intelligent Transportation Management

DA-RNN-Based Bus Arrival Time Prediction Model