Skip to main content
Erschienen in:

29.04.2022

Dynamic Weight-based Multi-Objective Reward Architecture for Adaptive Traffic Signal Control System

verfasst von: Abu Rafe Md Jamil, Naushin Nower

Erschienen in: International Journal of Intelligent Transportation Systems Research | Ausgabe 2/2022

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

An Adaptive Traffic Signal Control (ATSC) system uses real-time traffic information to control traffic lights and makes the public transport system more reliable and accessible. Deep Reinforcement Learning (DRL) has recently demonstrated its use in resolving traffic signal control problems. However, designing a good reward function is one of the most crucial aspects of DRL since the system learns to make proper decisions based on reward. Furthermore, the multi-objective reward function is preferable for the ATSC system, which is more challenging than designing a single objective. The existing multi-objective reward functions use pre-defined fixed weights to combine the multiple parameters, which requires rigorous training and cannot represent the actual impact of the parameters. To solve this problem, we proposed a new reward architecture called Dynamic Weights Multi-objective Reward Architecture (DWMORA) for ATSC. It calculates the weights instantly based on the current traffic condition to ensure the actual impact of the parameters. A comparative result study of the proposed approach with several existing reward functions shows the improvement of the road traffic in terms of waiting time, travel time, and halting number.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

ATZelectronics worldwide

ATZlectronics worldwide is up-to-speed on new trends and developments in automotive electronics on a scientific level with a high depth of information. 

Order your 30-days-trial for free and without any commitment.

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Literatur
1.
Zurück zum Zitat Abels, A., Roijers, D., Lenaerts, T., Nowé, A., Steckelmacher, D.: Dynamic weights in multi-objective deep reinforcement learning. In: International Conference on Machine Learning, pp. 11–20. PMLR (2019) Abels, A., Roijers, D., Lenaerts, T., Nowé, A., Steckelmacher, D.: Dynamic weights in multi-objective deep reinforcement learning. In: International Conference on Machine Learning, pp. 11–20. PMLR (2019)
2.
Zurück zum Zitat Brys, T., Harutyunyan, A., Vrancx, P., Taylor, M.E., Kudenko, D., Nowé, A.: Multi-objectivization of reinforcement learning problems by reward shaping. In: 2014 international joint conference on neural networks (IJCNN), pp. 2315–2322. IEEE (2014) Brys, T., Harutyunyan, A., Vrancx, P., Taylor, M.E., Kudenko, D., Nowé, A.: Multi-objectivization of reinforcement learning problems by reward shaping. In: 2014 international joint conference on neural networks (IJCNN), pp. 2315–2322. IEEE (2014)
3.
Zurück zum Zitat Das, I., Dennis, J.E.: A closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Structural optimization 14(1), 63–69 (1997)CrossRef Das, I., Dennis, J.E.: A closer look at drawbacks of minimizing weighted sums of objectives for pareto set generation in multicriteria optimization problems. Structural optimization 14(1), 63–69 (1997)CrossRef
4.
Zurück zum Zitat Hossan, S., Nower, N.: Fog-based dynamic traffic light control system for improving public transport. Public Transport pp. 1–24 (2020) Hossan, S., Nower, N.: Fog-based dynamic traffic light control system for improving public transport. Public Transport pp. 1–24 (2020)
5.
Zurück zum Zitat Jamil, A.R.M., Ganguly, K.K., Nower, N.: Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning. IET Intel. Transport Syst. 14(14), 2030–2041 (2021)CrossRef Jamil, A.R.M., Ganguly, K.K., Nower, N.: Adaptive traffic signal control system using composite reward architecture based deep reinforcement learning. IET Intel. Transport Syst. 14(14), 2030–2041 (2021)CrossRef
6.
Zurück zum Zitat Khamis, M.A., Gomaa, W.: Enhanced multiagent multi-objective reinforcement learning for urban traffic light control. In: 2012 11th International Conference on Machine Learning and Applications, vol. 1, pp. 586–591. IEEE (2012) Khamis, M.A., Gomaa, W.: Enhanced multiagent multi-objective reinforcement learning for urban traffic light control. In: 2012 11th International Conference on Machine Learning and Applications, vol. 1, pp. 586–591. IEEE (2012)
7.
Zurück zum Zitat Khamis, M.A., Gomaa, W.: Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. Eng. Appl. Artif. Intell. 29, 134–151 (2014)CrossRef Khamis, M.A., Gomaa, W.: Adaptive multi-objective reinforcement learning with hybrid exploration for traffic signal control based on cooperative multi-agent framework. Eng. Appl. Artif. Intell. 29, 134–151 (2014)CrossRef
8.
Zurück zum Zitat Khamis, M.A., Gomaa, W., El-Shishiny, H.: Multi-objective traffic light control system based on bayesian probability interpretation. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, pp. 995–1000. IEEE (2012) Khamis, M.A., Gomaa, W., El-Shishiny, H.: Multi-objective traffic light control system based on bayesian probability interpretation. In: 2012 15th International IEEE Conference on Intelligent Transportation Systems, pp. 995–1000. IEEE (2012)
9.
Zurück zum Zitat Krajzewicz, D., Erdmann, J., Behrisch, M., Bieker, L.: Recent development and applications of sumo-simulation of urban mobility. International Journal On Advances in Systems and Measurements 5(3spsampsps4),(2012) Krajzewicz, D., Erdmann, J., Behrisch, M., Bieker, L.: Recent development and applications of sumo-simulation of urban mobility. International Journal On Advances in Systems and Measurements 5(3spsampsps4),(2012)
10.
Zurück zum Zitat Lin, Y., Dai, X., Li, L., Wang, F.Y.: An efficient deep reinforcement learning model for urban traffic control. arXiv preprint arXiv:1808.01876 (2018) Lin, Y., Dai, X., Li, L., Wang, F.Y.: An efficient deep reinforcement learning model for urban traffic control. arXiv preprint arXiv:1808.01876 (2018)
11.
Zurück zum Zitat Van der Pol, E., Oliehoek, F.A.: Coordinated deep reinforcement learners for traffic light control. Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016) (2016) Van der Pol, E., Oliehoek, F.A.: Coordinated deep reinforcement learners for traffic light control. Proceedings of Learning, Inference and Control of Multi-Agent Systems (at NIPS 2016) (2016)
12.
Zurück zum Zitat Scorecard, U.M.: The texas a&m transportation institute and inrix. Inc., USA 9(2015), 10 (2015) Scorecard, U.M.: The texas a&m transportation institute and inrix. Inc., USA 9(2015), 10 (2015)
13.
Zurück zum Zitat Sutton, R.S., Barto, A.G., et al.: Introduction to reinforcement learning, vol. 135. MIT press Cambridge (1998) Sutton, R.S., Barto, A.G., et al.: Introduction to reinforcement learning, vol. 135. MIT press Cambridge (1998)
14.
Zurück zum Zitat Van Moffaert, K., Brys, T., Chandra, A., Esterle, L., Lewis, P.R., Nowé, A.: A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning. In: 2014 International joint conference on neural networks (IJCNN), pp. 2306–2314. IEEE (2014) Van Moffaert, K., Brys, T., Chandra, A., Esterle, L., Lewis, P.R., Nowé, A.: A novel adaptive weight selection algorithm for multi-objective multi-agent reinforcement learning. In: 2014 International joint conference on neural networks (IJCNN), pp. 2306–2314. IEEE (2014)
15.
Zurück zum Zitat Van Moffaert, K., Drugan, M.M., Nowé, A.: Scalarized multi-objective reinforcement learning: Novel design techniques. In: 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 191–199. IEEE (2013) Van Moffaert, K., Drugan, M.M., Nowé, A.: Scalarized multi-objective reinforcement learning: Novel design techniques. In: 2013 IEEE Symposium on Adaptive Dynamic Programming and Reinforcement Learning (ADPRL), pp. 191–199. IEEE (2013)
16.
Zurück zum Zitat Van Seijen, H., Fatemi, M., Romoff, J., Laroche, R., Barnes, T., Tsang, J.: Hybrid reward architecture for reinforcement learning. In: Advances in Neural Information Processing Systems, pp. 5392–5402 (2017) Van Seijen, H., Fatemi, M., Romoff, J., Laroche, R., Barnes, T., Tsang, J.: Hybrid reward architecture for reinforcement learning. In: Advances in Neural Information Processing Systems, pp. 5392–5402 (2017)
17.
Zurück zum Zitat Vidali, A., Crociani, L., Vizzari, G., Bandini, S.: A deep reinforcement learning approach to adaptive traffic lights management. In: Proceedings of the 20th Workshop” From Objects to Agents”, Parma, Italy (2019) Vidali, A., Crociani, L., Vizzari, G., Bandini, S.: A deep reinforcement learning approach to adaptive traffic lights management. In: Proceedings of the 20th Workshop” From Objects to Agents”, Parma, Italy (2019)
18.
Zurück zum Zitat Wei, H., Chen, C., Zheng, G., Wu, K., Gayah, V., Xu, K., Li, Z.: Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1290–1298 (2019) Wei, H., Chen, C., Zheng, G., Wu, K., Gayah, V., Xu, K., Li, Z.: Presslight: Learning max pressure control to coordinate traffic signals in arterial network. In: Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1290–1298 (2019)
19.
Zurück zum Zitat Wei, H., Zheng, G., Yao, H., Li, Z.: Intellilight: A reinforcement learning approach for intelligent traffic light control. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505 (2018) Wei, H., Zheng, G., Yao, H., Li, Z.: Intellilight: A reinforcement learning approach for intelligent traffic light control. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2496–2505 (2018)
20.
Zurück zum Zitat Xu, J., Tian, Y., Ma, P., Rus, D., Sueda, S., Matusik, W.: Prediction-guided multi-objective reinforcement learning for continuous robot control. In: International Conference on Machine Learning, pp. 10607–10616. PMLR (2020) Xu, J., Tian, Y., Ma, P., Rus, D., Sueda, S., Matusik, W.: Prediction-guided multi-objective reinforcement learning for continuous robot control. In: International Conference on Machine Learning, pp. 10607–10616. PMLR (2020)
21.
Zurück zum Zitat Yang, R., Sun, X., Narasimhan, K.: A generalized algorithm for multi-objective reinforcement learning and policy adaptation. arXiv preprint arXiv:1908.08342 (2019) Yang, R., Sun, X., Narasimhan, K.: A generalized algorithm for multi-objective reinforcement learning and policy adaptation. arXiv preprint arXiv:1908.08342 (2019)
Metadaten
Titel
Dynamic Weight-based Multi-Objective Reward Architecture for Adaptive Traffic Signal Control System
verfasst von
Abu Rafe Md Jamil
Naushin Nower
Publikationsdatum
29.04.2022
Verlag
Springer US
Erschienen in
International Journal of Intelligent Transportation Systems Research / Ausgabe 2/2022
Print ISSN: 1348-8503
Elektronische ISSN: 1868-8659
DOI
https://doi.org/10.1007/s13177-022-00305-5

    Marktübersichten

    Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.