Skip to main content
Erschienen in: Neural Processing Letters 1/2023

05.10.2022

Event-triggered multi-agent credit allocation pursuit-evasion algorithm

verfasst von: Bo-Kun Zhang, Bin Hu, Ding-Xue Zhang, Zhi-Hong Guan, Xin-Ming Cheng

Erschienen in: Neural Processing Letters | Ausgabe 1/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

The reinforcement learning is used to study the problem of multi-agent pursuit-evasion games in this article. The main problem of current reinforcement learning applied to multi-agents is the low learning efficiency of agents. To solve this problem, a credit allocation mechanism is adopted in the Multi-agent Deep Deterministic Policy Gradient frame (hereinafter referred to as the MADDPG), the core idea of which is to enable individuals who contribute more to the group to occupy a higher degree of dominance in subsequent training iterations. An event-triggered mechanism is utilized for the simplification of calculation. An observer is set for the feedback value, and the credit allocation algorithm is activated only when the observer believes that the agent group is in a local optimal training dilemma. The final simulation and experiment show that, In most cases, the event-triggered multiagent credit allocation algorithm (hereinafter referred to as the EDMCA algorithm) obtained better results and discussed the parameter settings of the observer.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Mnih V, Kavukcuoglu K, Silver D et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529–533.CrossRef Mnih V, Kavukcuoglu K, Silver D et al. (2015) Human-level control through deep reinforcement learning. Nature 518:529–533.CrossRef
2.
Zurück zum Zitat Ferber J, Weiss G (1999) Multi-agent systems: an introduction to distributed artificial intelligence. Reading, Addison-Wesley. Ferber J, Weiss G (1999) Multi-agent systems: an introduction to distributed artificial intelligence. Reading, Addison-Wesley.
4.
Zurück zum Zitat Singh S, Cohn D (1998) How to dynamically merge markov decision processes. Adv Neural Inf Process Syst 10:1057–1063. Singh S, Cohn D (1998) How to dynamically merge markov decision processes. Adv Neural Inf Process Syst 10:1057–1063.
5.
Zurück zum Zitat Tan M (1993) Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning 330–337 Tan M (1993) Multi-agent reinforcement learning: Independent vs. cooperative agents. Proceedings of the tenth international conference on machine learning 330–337
6.
Zurück zum Zitat Dayan P, Hinton GE (1993) Feudal reinforcement learning. Advances in neural information processing systems 271–278 Dayan P, Hinton GE (1993) Feudal reinforcement learning. Advances in neural information processing systems 271–278
7.
Zurück zum Zitat Vinyals O, Babuschkin I, Czarnecki WM et al. (2019) Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575:350–354.CrossRef Vinyals O, Babuschkin I, Czarnecki WM et al. (2019) Grandmaster level in StarCraft II using multi-agent reinforcement learning. Nature 575:350–354.CrossRef
8.
Zurück zum Zitat Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994:157–163. Littman ML (1994) Markov games as a framework for multi-agent reinforcement learning. Machine Learning Proceedings 1994:157–163.
9.
Zurück zum Zitat Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0) Sprague N, Ballard D (2003) Multiple-goal reinforcement learning with modular sarsa(0)
10.
Zurück zum Zitat Tesauro G (2004) Extending Q-learning to general adaptive multi-agent systems. Advances in neural information processing systems 16:871–878. Tesauro G (2004) Extending Q-learning to general adaptive multi-agent systems. Advances in neural information processing systems 16:871–878.
11.
Zurück zum Zitat Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30:6379–6390. Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. Advances in neural information processing systems 30:6379–6390.
12.
Zurück zum Zitat Foerster JN, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence 32:2974–2982.CrossRef Foerster JN, Farquhar G, Afouras T, Nardelli N, Whiteson S (2018) Counterfactual multi-agent policy gradients. Proceedings of the AAAI Conference on Artificial Intelligence 32:2974–2982.CrossRef
13.
Zurück zum Zitat Wang YW, Lei Y, Bian T, Guan ZH (2019) Distributed control of nonlinear multiagent systems with unknown and nonidentical control directions via event-triggered communication. IEEE Transactions on Cybernetics 50:1820–1832.CrossRef Wang YW, Lei Y, Bian T, Guan ZH (2019) Distributed control of nonlinear multiagent systems with unknown and nonidentical control directions via event-triggered communication. IEEE Transactions on Cybernetics 50:1820–1832.CrossRef
14.
Zurück zum Zitat Guan ZH, Hu B, Chi M, He DX, Cheng XM (2014) Guaranteed performance consensus in second-order multi-agent systems with hybrid impulsive control. Automatica 50:2415–2418.MathSciNetCrossRefMATH Guan ZH, Hu B, Chi M, He DX, Cheng XM (2014) Guaranteed performance consensus in second-order multi-agent systems with hybrid impulsive control. Automatica 50:2415–2418.MathSciNetCrossRefMATH
15.
Zurück zum Zitat Guan ZH, Hill DJ, Shen X (2005) On hybrid impulsive and switching systems and application to nonlinear control. IEEE Trans Autom Control 50:1058–1062.MathSciNetCrossRefMATH Guan ZH, Hill DJ, Shen X (2005) On hybrid impulsive and switching systems and application to nonlinear control. IEEE Trans Autom Control 50:1058–1062.MathSciNetCrossRefMATH
16.
Zurück zum Zitat Foerster J, Nardelli N, Farquhar G, Afouras T, Torr P, Kohli P, Whiteson S (2017) Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 34th International Conference on Machine Learning 70:1146–1155 Foerster J, Nardelli N, Farquhar G, Afouras T, Torr P, Kohli P, Whiteson S (2017) Stabilising experience replay for deep multi-agent reinforcement learning. Proceedings of the 34th International Conference on Machine Learning 70:1146–1155
17.
18.
Zurück zum Zitat Yang YD, Luo R, Li M, Zhou M, Zhang WN, Wang J (2018) Mean field multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning 80:5571–5580 Yang YD, Luo R, Li M, Zhou M, Zhang WN, Wang J (2018) Mean field multi-agent reinforcement learning. Proceedings of the 35th International Conference on Machine Learning 80:5571–5580
19.
Zurück zum Zitat Omidshafiei S, Pazis J, Amato C, How JP, Vian J (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. Proceedings of the 34th International Conference on Machine Learning 70:2681–2690 Omidshafiei S, Pazis J, Amato C, How JP, Vian J (2017) Deep decentralized multi-task multi-agent reinforcement learning under partial observability. Proceedings of the 34th International Conference on Machine Learning 70:2681–2690
20.
Zurück zum Zitat Gupta J K, Egorov M, Kochenderfer M (2017) Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems 66–83 Gupta J K, Egorov M, Kochenderfer M (2017) Cooperative multi-agent control using deep reinforcement learning. International Conference on Autonomous Agents and Multiagent Systems 66–83
21.
Zurück zum Zitat Foerster JN, Assael YM, Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. arXiv:1605.06676 Foerster JN, Assael YM, Freitas N, Whiteson S (2016) Learning to communicate with deep multi-agent reinforcement learning. arXiv:​1605.​06676
22.
Zurück zum Zitat Ghosh A, Kulharia V, Namboodiri VP, Torr P, Dokania PK (2018) Multi-agent diverse generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8513–8521 Ghosh A, Kulharia V, Namboodiri VP, Torr P, Dokania PK (2018) Multi-agent diverse generative adversarial networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8513–8521
24.
Zurück zum Zitat Iqbal S, Sha F (2019) Actor-attention-critic for multi-agent reinforcement learning. International Conference on Machine Learning 2961–2970 Iqbal S, Sha F (2019) Actor-attention-critic for multi-agent reinforcement learning. International Conference on Machine Learning 2961–2970
25.
Zurück zum Zitat Liu K, Duan P, Duan Z, Cai H, Lü J (2018) Leader-following consensus of multi-agent systems with switching networks and event-triggered control. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:1696–1706.CrossRef Liu K, Duan P, Duan Z, Cai H, Lü J (2018) Leader-following consensus of multi-agent systems with switching networks and event-triggered control. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:1696–1706.CrossRef
26.
Zurück zum Zitat Wu ZG, Xu Y, Pan YJ, Su H, Tang Y (2018) Event-triggered control for consensus problem in multi-agent systems with quantized relative state measurements and external disturbance. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:2232–2242.MathSciNetCrossRefMATH Wu ZG, Xu Y, Pan YJ, Su H, Tang Y (2018) Event-triggered control for consensus problem in multi-agent systems with quantized relative state measurements and external disturbance. IEEE Transactions on Circuits and Systems-I: Regular Papers 65:2232–2242.MathSciNetCrossRefMATH
27.
Zurück zum Zitat Luo S, Ye D (2019) Adaptive double event-triggered control for linear multi-agent systems with actuator faults. IEEE Transactions on Circuits and Systems-I: Regular Papers 66:4829–4839.MathSciNetCrossRefMATH Luo S, Ye D (2019) Adaptive double event-triggered control for linear multi-agent systems with actuator faults. IEEE Transactions on Circuits and Systems-I: Regular Papers 66:4829–4839.MathSciNetCrossRefMATH
28.
Zurück zum Zitat Yang X, Zhu Q (2021) Stabilization of stochastic retarded systems based on sampled-data feedback control. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51:5895–5904.CrossRef Yang X, Zhu Q (2021) Stabilization of stochastic retarded systems based on sampled-data feedback control. IEEE Transactions on Systems, Man, and Cybernetics: Systems 51:5895–5904.CrossRef
29.
Zurück zum Zitat Zhu Q (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans Autom Control 64:3764–3771.MathSciNetCrossRefMATH Zhu Q (2019) Stabilization of stochastic nonlinear delay systems with exogenous disturbances and the event-triggered feedback control. IEEE Trans Autom Control 64:3764–3771.MathSciNetCrossRefMATH
Metadaten
Titel
Event-triggered multi-agent credit allocation pursuit-evasion algorithm
verfasst von
Bo-Kun Zhang
Bin Hu
Ding-Xue Zhang
Zhi-Hong Guan
Xin-Ming Cheng
Publikationsdatum
05.10.2022
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 1/2023
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-022-10909-3

Weitere Artikel der Ausgabe 1/2023

Neural Processing Letters 1/2023 Zur Ausgabe

Neuer Inhalt