Skip to main content
Top
Published in: Neural Processing Letters 1/2023

05-10-2022

Event-triggered multi-agent credit allocation pursuit-evasion algorithm

Authors: Bo-Kun Zhang, Bin Hu, Ding-Xue Zhang, Zhi-Hong Guan, Xin-Ming Cheng

Published in: Neural Processing Letters | Issue 1/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The reinforcement learning is used to study the problem of multi-agent pursuit-evasion games in this article. The main problem of current reinforcement learning applied to multi-agents is the low learning efficiency of agents. To solve this problem, a credit allocation mechanism is adopted in the Multi-agent Deep Deterministic Policy Gradient frame (hereinafter referred to as the MADDPG), the core idea of which is to enable individuals who contribute more to the group to occupy a higher degree of dominance in subsequent training iterations. An event-triggered mechanism is utilized for the simplification of calculation. An observer is set for the feedback value, and the credit allocation algorithm is activated only when the observer believes that the agent group is in a local optimal training dilemma. The final simulation and experiment show that, In most cases, the event-triggered multiagent credit allocation algorithm (hereinafter referred to as the EDMCA algorithm) obtained better results and discussed the parameter settings of the observer.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Mnih V, Kavukcuoglu K, Silver D et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529–533.CrossRef Mnih V, Kavukcuoglu K, Silver D et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529–533.CrossRef
2.
go back to reference Ferber J, Weiss G (1999) Multi-agent systems: an introduction to distributed artificial intelligence. Reading, Addison-Wesley. Ferber J, Weiss G (1999) Multi-agent systems: an introduction to distributed artificial intelligence. Reading, Addison-Wesley.
4.
go back to reference Singh S, Cohn D (1998) How to dynamically merge markov decision processes. Adv Neural Inf Process Syst 10:1057–1063. Singh S, Cohn D (1998) How to dynamically merge markov decision processes. Adv Neural Inf Process Syst 10:1057–1063.
5.
go back to reference Tan M (1993) Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning 330–337 Tan M (1993) Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning 330–337
6.
go back to reference Dayan P, Hinton GE (1993) Feudal reinforcement learning. Advances in neural information processing systems 271–278 Dayan P, Hinton GE (1993) Feudal reinforcement learning. Advances in neural information processing systems 271–278
7.
go back to reference Vinyals O, Babuschkin I, Czarnecki WM et al. (2019) Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575:350–354.CrossRef Vinyals O, Babuschkin I, Czarnecki WM et al. (2019) Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575:350–354.CrossRef
8.
go back to reference Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994:157–163. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994:157–163.
9.
go back to reference Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0) Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0)
10.
go back to reference Tesauro G (2004) Extending Q-learning to general adaptive multi-agent systems. Advances in neural information processing systems 16:871–878. Tesauro G (2004) Extending Q-learning to general adaptive multi-agent systems. Advances in neural information processing systems 16:871–878.
11.
go back to reference Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30:6379–6390. Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30:6379–6390.
12.
go back to reference Foerster JN, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence 32:2974–2982.CrossRef Foerster JN, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence 32:2974–2982.CrossRef
13.
go back to reference Wang YW, Lei Y, Bian T, Guan ZH (2019) Distributed control of nonlinear multiagent systems with unknown and nonidentical control directions via event-triggered communication. IEEE Transactions on Cybernetics 50:1820–1832.CrossRef Wang YW, Lei Y, Bian T, Guan ZH (2019) Distributed control of nonlinear multiagent systems with unknown and nonidentical control directions via event-triggered communication. IEEE Transactions on Cybernetics 50:1820–1832.CrossRef
14.
go back to reference Guan ZH, Hu B, Chi M, He DX, Cheng XM (2014) Guaranteed performance consensus in second-order multi-agent systems with hybrid impulsive control. Automatica 50:2415–2418.MathSciNetCrossRefMATH Guan ZH, Hu B, Chi M, He DX, Cheng XM (2014) Guaranteed performance consensus in second-order multi-agent systems with hybrid impulsive control. Automatica 50:2415–2418.MathSciNetCrossRefMATH
15.
go back to reference Guan ZH, Hill DJ, Shen X (2005) On hybrid impulsive and switching systems and application to nonlinear control. IEEE Trans Autom Control 50:1058–1062.MathSciNetCrossRefMATH Guan ZH, Hill DJ, Shen X (2005) On hybrid impulsive and switching systems and application to nonlinear control. IEEE Trans Autom Control 50:1058–1062.MathSciNetCrossRefMATH
16.
go back to reference Foerster J, Nardelli N, Farquhar G, Afouras T, Torr P, Kohli P, Whiteson S (2017) Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 34th International Conference on Machine Learning 70:1146–1155 Foerster J, Nardelli N, Farquhar G, Afouras T, Torr P, Kohli P, Whiteson S (2017) Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 34th International Conference on Machine Learning 70:1146–1155
17.
18.
go back to reference Yang YD, Luo R, Li M, Zhou M, Zhang WN, Wang J (2018) Mean field multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning 80:5571–5580 Yang YD, Luo R, Li M, Zhou M, Zhang WN, Wang J (2018) Mean field multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning 80:5571–5580
19.
go back to reference Omidshafiei S, Pazis J, Amato C, How JP, Vian J (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. Proceedings of the 34th International Conference on Machine Learning 70:2681–2690 Omidshafiei S, Pazis J, Amato C, How JP, Vian J (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. Proceedings of the 34th International Conference on Machine Learning 70:2681–2690
20.
go back to reference Gupta J K, Egorov M, Kochenderfer M (2017) Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems 66–83 Gupta J K, Egorov M, Kochenderfer M (2017) Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems 66–83
21.
go back to reference Foerster JN, Assael YM, Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. arXiv:1605.06676 Foerster JN, Assael YM, Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. arXiv:​1605.​06676
22.
go back to reference Ghosh A, Kulharia V, Namboodiri VP, Torr P, Dokania PK (2018) Multi-agent diverse generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8513–8521 Ghosh A, Kulharia V, Namboodiri VP, Torr P, Dokania PK (2018) Multi-agent diverse generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8513–8521
24.
go back to reference Iqbal S, Sha F (2019) Actor-attention-critic for multi-agent reinforcement learning. International Conference on Machine Learning 2961–2970 Iqbal S, Sha F (2019) Actor-attention-critic for multi-agent reinforcement learning. International Conference on Machine Learning 2961–2970
25.
go back to reference Liu K, Duan P, Duan Z, Cai H, Lü J (2018) Leader-following consensus of multi-agent systems with switching networks and event-triggered control. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:1696–1706.CrossRef Liu K, Duan P, Duan Z, Cai H, Lü J (2018) Leader-following consensus of multi-agent systems with switching networks and event-triggered control. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:1696–1706.CrossRef
26.
go back to reference Wu ZG, Xu Y, Pan YJ, Su H, Tang Y (2018) Event-triggered control for consensus problem in multi-agent systems with quantized relative state measurements and external disturbance. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:2232–2242.MathSciNetCrossRefMATH Wu ZG, Xu Y, Pan YJ, Su H, Tang Y (2018) Event-triggered control for consensus problem in multi-agent systems with quantized relative state measurements and external disturbance. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:2232–2242.MathSciNetCrossRefMATH
27.
go back to reference Luo S, Ye D (2019) Adaptive double event-triggered control for linear multi-agent systems with actuator faults. IEEE Transactions on Circuits and Systems-I: Regular Papers 66:4829–4839.MathSciNetCrossRefMATH Luo S, Ye D (2019) Adaptive double event-triggered control for linear multi-agent systems with actuator faults. IEEE Transactions on Circuits and Systems-I: Regular Papers 66:4829–4839.MathSciNetCrossRefMATH
28.
go back to reference Yang X, Zhu Q (2021) Stabilization of stochastic retarded systems based on sampled-data feedback control. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51:5895–5904.CrossRef Yang X, Zhu Q (2021) Stabilization of stochastic retarded systems based on sampled-data feedback control. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51:5895–5904.CrossRef
29.
go back to reference Zhu Q (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans Autom Control 64:3764–3771.MathSciNetCrossRefMATH Zhu Q (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans Autom Control 64:3764–3771.MathSciNetCrossRefMATH
Metadata
Title
Event-triggered multi-agent credit allocation pursuit-evasion algorithm
Authors
Bo-Kun Zhang
Bin Hu
Ding-Xue Zhang
Zhi-Hong Guan
Xin-Ming Cheng
Publication date
05-10-2022
Publisher
Springer US
Published in
Neural Processing Letters / Issue 1/2023
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10909-3

Other articles of this Issue 1/2023

Neural Processing Letters 1/2023 Go to the issue