Skip to main content
Top

2018 | OriginalPaper | Chapter

Intelligent Traffic Control by Multi-agent Cooperative Q Learning (MCQL)

Authors : Deepak A. Vidhate, Parag Kulkarni

Published in: Intelligent Computing and Information and Communication

Publisher: Springer Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Traffic crisis frequently happens because of traffic demands by the large number vehicles on the path. Increasing transportation move and decreasing the average waiting time of each vehicle are the objectives of cooperative intelligent traffic control system. Each signal wishes to catch better travel move. During the course, signals form a strategy of cooperation in addition to restriction for neighboring signals to exploit their individual benefit. A superior traffic signal scheduling strategy is useful to resolve the difficulty. The several parameters may influence the traffic control model. So it is hard to learn the best possible result. The lack of expertise of traffic light controllers to study from previous practice results makes them to be incapable of incorporating uncertain modifications of traffic flow. Defining instantaneous features of the real traffic scenario, reinforcement learning algorithm based traffic control model can be used to obtain fine timing rules. The projected real-time traffic control optimization model is able to continue with the traffic signal scheduling rules successfully. The model expands traffic value of the vehicle, which consists of delay time, the number of vehicles stopped at the signal, and the newly arriving vehicles to learn and establish the optimal actions. The experimentation outcome illustrates a major enhancement in traffic control, demonstrating the projected model is competent of making possible real-time dynamic traffic control.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference F. Zhu, J. Ning, Y. Ren, and J. Peng, “Optimization of image processing in video-based traffic monitoring,” ElektronikairElektrotechnika, vol.18, no.8, pp. 91–96, 2012. F. Zhu, J. Ning, Y. Ren, and J. Peng, “Optimization of image processing in video-based traffic monitoring,” ElektronikairElektrotechnika, vol.18, no.8, pp. 91–96, 2012.
2.
go back to reference B. de Schutter, “Optimal traffic light control for a single intersection,” in Proceedings of the American Control Conference (ACC ’99), vol. 3, pp. 2195–2199, June 1999. B. de Schutter, “Optimal traffic light control for a single intersection,” in Proceedings of the American Control Conference (ACC ’99), vol. 3, pp. 2195–2199, June 1999.
3.
go back to reference N. Findler and J. Stapp,“A distributed approach to optimized control of street traffic signals,” Journal of Transportation Engineering, vol.118, no.1, pp. 99–110, 1992. N. Findler and J. Stapp,“A distributed approach to optimized control of street traffic signals,” Journal of Transportation Engineering, vol.118, no.1, pp. 99–110, 1992.
4.
go back to reference L. D. Baskar and H. Hellendoorn, “Traffic management for automated highway systems using model-based control,”IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 2, pp. 838–847, 2012. L. D. Baskar and H. Hellendoorn, “Traffic management for automated highway systems using model-based control,”IEEE Transactions on Intelligent Transportation Systems, vol. 3, no. 2, pp. 838–847, 2012.
5.
go back to reference R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, Mass, USA, 1998. R. S. Sutton and A. G. Barto, Reinforcement Learning: An Introduction, MIT Press, Cambridge, Mass, USA, 1998.
6.
go back to reference Artificial Intelligence in Transportation Information for Application, Transportation Research CIRCULAR, Number E–C 113, Transportation On Research Board of the National Academies, January 2007. Artificial Intelligence in Transportation Information for Application, Transportation Research CIRCULAR, Number E–C 113, Transportation On Research Board of the National Academies, January 2007.
7.
go back to reference Deepak A. Vidhate, Parag Kulkarni “New Approach for Advanced Cooperative Learning Algorithms using RL methods (ACLA)” VisionNet’16 Proceedings of the Third International Symposium on Computer Vision and the Internet, ACM DL pp 12–20, 2016. Deepak A. Vidhate, Parag Kulkarni “New Approach for Advanced Cooperative Learning Algorithms using RL methods (ACLA)” VisionNet’16 Proceedings of the Third International Symposium on Computer Vision and the Internet, ACM DL pp 12–20, 2016.
8.
go back to reference K. Mase and H. Yamamoto, “Advanced traffic control methods for network management,” IEEE Magazine, vol. 28, no. 10, pp. 82–88, 1990. K. Mase and H. Yamamoto, “Advanced traffic control methods for network management,” IEEE Magazine, vol. 28, no. 10, pp. 82–88, 1990.
9.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Innovative Approach Towards Cooperation Models for Multi-agent Reinforcement Learning (CMMARL)” in Smart Trends in Information Technology and Computer Communications, Springer Nature, Vol 628, pp 468–478, 2016. Deepak A. Vidhate, Parag Kulkarni “Innovative Approach Towards Cooperation Models for Multi-agent Reinforcement Learning (CMMARL)” in Smart Trends in Information Technology and Computer Communications, Springer Nature, Vol 628, pp 468–478, 2016.
10.
go back to reference L. D. Baskar, B. de Schutter, J. Hellendoorn, and Z. Papp, “Traffic control and intelligent vehicle highway systems: a survey,” IET Intelligent Transport Systems, vol. 5, no. 1, pp. 38–52, 2011. L. D. Baskar, B. de Schutter, J. Hellendoorn, and Z. Papp, “Traffic control and intelligent vehicle highway systems: a survey,” IET Intelligent Transport Systems, vol. 5, no. 1, pp. 38–52, 2011.
11.
go back to reference M. Broucke “A theory of traffic flow in automated highway systems,” Transportation Research C, vol. 4, no. 4, pp. 181–210, 1996. M. Broucke “A theory of traffic flow in automated highway systems,” Transportation Research C, vol. 4, no. 4, pp. 181–210, 1996.
12.
go back to reference D. Helbing, A. Hennecke, V. Shvetsov, and M. Treiber, “Micro and macro-simulation of freeway traffic,” Mathematical and Computer Modelling, vol. 35, no. 5–6, pp. 517–547, 2002. D. Helbing, A. Hennecke, V. Shvetsov, and M. Treiber, “Micro and macro-simulation of freeway traffic,” Mathematical and Computer Modelling, vol. 35, no. 5–6, pp. 517–547, 2002.
13.
go back to reference S. Zegeye, B. de Schutter, J. Hellendoorn, E. A. Breunesse, and A. Hegyi, “A predictive traffic controller for sustainable mobility using parameterized control policies,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 3, pp. 1420–1429, 2012. S. Zegeye, B. de Schutter, J. Hellendoorn, E. A. Breunesse, and A. Hegyi, “A predictive traffic controller for sustainable mobility using parameterized control policies,” IEEE Transactions on Intelligent Transportation Systems, vol. 13, no. 3, pp. 1420–1429, 2012.
14.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Enhancement in Decision Making with Improved Performance by Multiagent Learning Algorithms” IOSR Journal of Computer Engineering, Volume 1, Issue 18, pp 18–25, 2016. Deepak A. Vidhate, Parag Kulkarni “Enhancement in Decision Making with Improved Performance by Multiagent Learning Algorithms” IOSR Journal of Computer Engineering, Volume 1, Issue 18, pp 18–25, 2016.
15.
go back to reference A. Bonarini and M. Restelli, “Reinforcement distribution in fuzzy Q-learning,” Fuzzy Sets and Systems, vol.160, no.10, pp. 1420–1443, 2009. A. Bonarini and M. Restelli, “Reinforcement distribution in fuzzy Q-learning,” Fuzzy Sets and Systems, vol.160, no.10, pp. 1420–1443, 2009.
16.
go back to reference Y. K. Chin, Y. K. Wei, and K. T. K. Teo, “Qlearning traffic signal optimization within multiple intersections traffic network,” in Proceedings of the 6th UKSim/AMSS European Symposium on Computer Modeling and Simulation (EMS ’12), pp. 343–348, Nov 2012. Y. K. Chin, Y. K. Wei, and K. T. K. Teo, “Qlearning traffic signal optimization within multiple intersections traffic network,” in Proceedings of the 6th UKSim/AMSS European Symposium on Computer Modeling and Simulation (EMS ’12), pp. 343–348, Nov 2012.
17.
go back to reference L.A. Prashanth and S. Bhatnagar, “Reinforcement learning with function approximation for traffic signal control,” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 412–421, 2011. L.A. Prashanth and S. Bhatnagar, “Reinforcement learning with function approximation for traffic signal control,” IEEE Transactions on Intelligent Transportation Systems, vol. 12, no. 2, pp. 412–421, 2011.
18.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Multilevel Relationship Algorithm for Association Rule Mining used for Cooperative Learning” in International Journal of Computer Applications (IJCA), Volume 86 Number 4- 2014 pp. 20–27. Deepak A. Vidhate, Parag Kulkarni “Multilevel Relationship Algorithm for Association Rule Mining used for Cooperative Learning” in International Journal of Computer Applications (IJCA), Volume 86 Number 4- 2014 pp. 20–27.
19.
go back to reference Y. K. Chin, L. K. Lee, N. Bolong, S. S. Yang, and K. T. K. Teo, “Exploring Q-learning optimization in traffic signal timing plan management,” in Proceedings of the 3rd International Conference on Computational Intelligence, Communication Systems and Networks (CICSyN ’11), pp. 269–274, July 2011. Y. K. Chin, L. K. Lee, N. Bolong, S. S. Yang, and K. T. K. Teo, “Exploring Q-learning optimization in traffic signal timing plan management,” in Proceedings of the 3rd International Conference on Computational Intelligence, Communication Systems and Networks (CICSyN ’11), pp. 269–274, July 2011.
20.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Multi-agent Cooperation Methods by Reinforcement Learning (MCMRL)”, Elsevier International Conference on Advanced Material Technologies (ICAMT)-2016}No. SS-LTMLBDA-06-05, 2016. Deepak A. Vidhate, Parag Kulkarni “Multi-agent Cooperation Methods by Reinforcement Learning (MCMRL)”, Elsevier International Conference on Advanced Material Technologies (ICAMT)-2016}No. SS-LTMLBDA-06-05, 2016.
21.
go back to reference S. Russell and P. Norvi, Artificial Intelligence: A Modern Approach, PHI, 2009. S. Russell and P. Norvi, Artificial Intelligence: A Modern Approach, PHI, 2009.
22.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Performance enhancement of cooperative learning algorithms by improved decision making for context based application”, International Conference on Automatic Control and Dynamic Optimization Techniques (ICACDOT) IEEE Xplorer, pp 246–252, 2016. Deepak A. Vidhate, Parag Kulkarni “Performance enhancement of cooperative learning algorithms by improved decision making for context based application”, International Conference on Automatic Control and Dynamic Optimization Techniques (ICACDOT) IEEE Xplorer, pp 246–252, 2016.
23.
go back to reference Deepak A. Vidhate, Parag Kulkarni “Improvement In Association Rule Mining By Multilevel Relationship algorithm” in International Journal of Research in Advent Technology (IJRAT), Volume 2 Number 1- 2014 pp. 366–373. Deepak A. Vidhate, Parag Kulkarni “Improvement In Association Rule Mining By Multilevel Relationship algorithm” in International Journal of Research in Advent Technology (IJRAT), Volume 2 Number 1- 2014 pp. 366–373.
24.
go back to reference Young-Cheol Choi, Student Member, Hyo-Sung Ahn “A Survey on Multi-Agent Reinforcement Learning: Coordination Problems”, IEEE/ASME International Conference on Mechatronics Embedded Systems and Applications, pp. 81–86, 2010. Young-Cheol Choi, Student Member, Hyo-Sung Ahn “A Survey on Multi-Agent Reinforcement Learning: Coordination Problems”, IEEE/ASME International Conference on Mechatronics Embedded Systems and Applications, pp. 81–86, 2010.
Metadata
Title
Intelligent Traffic Control by Multi-agent Cooperative Q Learning (MCQL)
Authors
Deepak A. Vidhate
Parag Kulkarni
Copyright Year
2018
Publisher
Springer Singapore
DOI
https://doi.org/10.1007/978-981-10-7245-1_47

Premium Partner