Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
International Journal of Machine Learning and Cybernetics
Published in: International Journal of Machine Learning and Cybernetics 6/2023

27-12-2022 | Original Article

Twin attentive deep reinforcement learning for multi-agent defensive convoy

Authors: Dongyu Fan, Haikuo Shen, Lijing Dong

Published in: International Journal of Machine Learning and Cybernetics | Issue 6/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Multi-agent defensive convoy helps provide critical safety for a leader agent. Escort agents work by coordinating their actions to protect the leader agent in the convoy. This paper investigates the multi-agent defensive convoy problem based on deep reinforcement learning and attention mechanism. To address the joint overestimation and suboptimal policy in multi-agent environments, a novel multi-agent twin attentive reinforcement learning method is proposed with a twin attentive critic and a delay attenuation policy. In addition, a variable temperature coefficient for maximum entropy is added to the learning process. The proposed method is evaluated on the designed defensive convoy environment and two public experimental environments, where our proposed method produces competitive performance compared to prior works. The contribution of each novel component is also extensively studied and analyzed. Further evaluations show that our method is robust to several adaptations in the defensive convoy environments including a changing number of escort agents and a changing number of dangers.
previous article Training fuzzy deep neural network with honey badger algorithm for intrusion detection in cloud environment
next article Incremental approaches for optimal scale selection in dynamic multi-scale set-valued decision tables

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
Show more products
Literature
1.
Kaur N, Kaur H (2022) A multi-agent based evacuation planning for disaster management: a narrative review. Arch Comput Methods Eng 29:4085–4113
2.
Ben-Dor G, Ben-Elia E, Benenson I (2021) Population downscaling in multi-agent transportation simulations: a review and case study. Simul Model Pract Theory 108:102233
3.
Amirkhani A, Barshooi AH (2021) Consensus in multi-agent systems: a review. Artif Intell Rev 55:3897–3935
4.
Mahmoud MS (2020) Multiagent systems: introduction and coordination control. CRC Press, Boca RatonMATH
5.
Hasan YA, Garg A, Sugaya S, Tapia L (2020) Defensive escort teams for navigation in crowds via multi-agent deep reinforcement learning. IEEE Robot Autom Lett 5(4):5645–5652
6.
Perrusqu’ia A, Yu W, Li X (2021) Multi-agent reinforcement learning for redundant robot control in task-space. Int J Mach Learn Cybern 12:231–241
7.
Ji G, Yan J, Du J, Yan W, Chen J, Lu Y, Rojas J, Cheng SS (2021) Towards safe control of continuum manipulator using shielded multiagent reinforcement learning. IEEE Robot Autom Lett 6(4):7461–7468
8.
Ren L, Fan X, Cui J, Shen Z, Lv Y, Xiong G (2022) A multi-agent reinforcement learning method with route recorders for vehicle routing in supply chain management. IEEE Trans Intell Transp Syst 23(9):16410–16420
9.
Kumar AS, Zhao L, Fernando X (2022) Multi-agent deep reinforcement learning-empowered channel allocation in vehicular networks. IEEE Trans Veh Technol 71(2):1726–1736
10.
Panerati J, Zheng H, Zhou S, Xu J, Prorok A, Schoellig AP (2021) Learning to fly-a gym environment with pybullet physics for reinforcement learning of multi-agent quadcopter control. In: 2021 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 7512–7519
11.
de Souza C, Newbury R, Cosgun A, Castillo P, Vidolov B, Kulić D (2021) Decentralized multi-agent pursuit using deep reinforcement learning. IEEE Robot Autom Lett 6(3):4552–4559
12.
Xia Z, Du J, Wang J, Jiang C, Ren Y, Li G, Han Z (2022) Multi-agent reinforcement learning aided intelligent UAV swarm for target tracking. IEEE Trans Veh Technol 71(1):931–945
13.
Sacco A, Esposito F, Marchetto G, Montuschi P (2021) Sustainable task offloading in UAV networks via multi-agent reinforcement learning. IEEE Trans Veh Technol 70(5):5003–5015
14.
Zhang H, Cheng J, Zhang L, Li Y, Zhang W (2022) H2GNN: hierarchical-hops graph neural networks for multi-robot exploration in unknown environments. IEEE Robot Autom Lett 7(2):3435–3442
15.
Xie J, Luo J, Peng Y, Xie S, Pu H, Li X, Su Z, Liu Y, Zhou R (2020) Data driven hybrid edge computing-based hierarchical task guidance for efficient maritime escorting with multiple unmanned surface vehicles. Peer-to-Peer Netw Appl 13(5):1788–1798
16.
Ma J, Lu H, Xiao J, Zeng Z, Zheng Z (2020) Multi-robot target encirclement control with collision avoidance via deep reinforcement learning. J Intell Robot Syst 99(2):371–386
17.
Gronauer S, Diepold K (2021) Multi-agent deep reinforcement learning: a survey. Artif Intell Rev 55:895–943
18.
Nguyen TT, Nguyen ND, Nahavandi S (2020) Deep reinforcement learning for multiagent systems: a review of challenges, solutions, and applications. IEEE Trans Cybern 50(9):3826–3839
19.
Sadhu AK, Konar A (2020) Multi-agent coordination: a reinforcement learning approach. Wiley, Hoboken
20.
Lyu X, Xiao Y, Daley B, Amato C (2021) Contrasting centralized and decentralized critics in multi-agent reinforcement learning. In: Proceedings of the 20th international conference on autonomous agents and multiagent systems, pp 844–852
21.
Du W, Ding S (2021) A survey on multi-agent deep reinforcement learning: from the perspective of challenges and applications. Artif Intell Rev 54(5):3215–3238
22.
Du W, Ding S, Zhang C, Du S (2021) Modified action decoder using Bayesian reasoning for multi-agent deep reinforcement learning. Int J Mach Learn Cybern 12(10):2947–2961
23.
Cao D, Zhao J, Hu W, Ding F, Huang Q, Chen Z, Blaabjerg F (2021) Data-driven multi-agent deep reinforcement learning for distribution system decentralized voltage control with high penetration of pvs. IEEE Trans Smart Grid 12(5):4137–4150
24.
Ye Z, Chen Y, Jiang X, Song G, Yang B, Fan S (2021) Improving sample efficiency in multi-agent actor-critic methods. Appl Intell 52:3691–3704
25.
Xu C, Liu S, Zhang C, Huang Y, Lu Z, Yang L (2021) Multi-agent reinforcement learning based distributed transmission in collaborative cloud-edge systems. IEEE Trans Veh Technol 70(2):1658–1672
26.
Lowe R, Wu Y, Tamar A, Harb J, Abbeel P, Mordatch I (2017) Multi-agent actor-critic for mixed cooperative-competitive environments. In: Advances in neural information processing systems 30 (NIPS), Long Beach, CA, USA, 4–9 December 2017, pp 6379–6390
27.
Zeng P, Cui S, Song C, Wang Z, Li G (2022) A multiagent deep deterministic policy gradient-based distributed protection method for distribution network. Neural Comput Appl
28.
Huang L, Fu M, Qu H, Wang S, Hu S (2021) A deep reinforcement learning-based method applied for solving multi-agent defense and attack problems. Expert Syst Appl 176:114896
29.
Chen X, Liu G (2021) Energy-efficient task offloading and resource allocation via deep reinforcement learning for augmented reality in mobile edge networks. IEEE Internet Things J 8(13):10843–10856
30.
Yang Y, Li B, Zhang S, Zhao W, Zhang H (2021) Cooperative proactive eavesdropping based on deep reinforcement learning. IEEE Wirel Commun Lett 10(9):1857–1861
31.
Wang L, Wang K, Pan C, Xu W, Aslam N, Hanzo L (2021) Multi-agent deep reinforcement learning-based trajectory planning for multi-UAV assisted mobile edge computing. IEEE Trans Cogn Commun Network 7(1):73–84
32.
Wu T, Zhou P, Wang B, Li A, Tang X, Xu Z, Chen K, Ding X (2021) Joint traffic control and multi-channel reassignment for core backbone network in SDN-IoT: a multi-agent deep reinforcement learning approach. IEEE Trans Netw Sci Eng 8(1):231–245MathSciNet
33.
Lillicrap TP, Hunt JJ, Pritzel A, Heess N, Erez T, Tassa Y, Silver D, Wierstra D (2016) Continuous control with deep reinforcement learning. In: 4th international conference on learning representations (ICLR), San Juan, Puerto Rico, May 2–4, 2016
34.
Mnih V, Kavukcuoglu K, Silver D et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529–533
35.
Hasselt Hv, Guez A, Silver D (2016) Deep reinforcement learning with double q-learning. In: Proceedings of the thirtieth AAAI conference on artificial intelligence, February 12–17, 2016, Phoenix, Arizona, USA, pp 2094–2100
36.
Fujimoto S, van Hoof H, Meger D (2018) Addressing function approximation error in actor-critic methods. In: Proceedings of the 35th international conference on machine learning (ICML), Stockholm Sweden, 10–15 July, 2018, vol 80, pp 1582–1591
37.
Zhang F, Li J, Li Z (2020) A TD3-based multi-agent deep reinforcement learning method in mixed cooperation-competition environment. Neurocomputing 411:206–215
38.
Chaudhuri K, Salakhutdinov R (2019) Actor-attention-critic for multi-agent reinforcement learning. In: Proceedings of the 36th international conference on machine learning (ICML), 9–15 June 2019, Long Beach, California, USA
39.
Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT Press, CambridgeMATH
40.
Gupta S, Singal G, Garg D (2021) Deep reinforcement learning techniques in diversified domains: a survey. Arch Comput Methods Eng 28:4715–4754
41.
Silver D, Huang A, Maddison C et al (2016) Mastering the game of go with deep neural networks and tree search. Nature 529:484–489
42.
Silver D, Schrittwieser J, Simonyan K et al (2017) Mastering the game of go without human knowledge. Nature 550:354–359
43.
Jin Z, Wu J, Liu A, Zhang W-A, Yu L (2022) Policy-based deep reinforcement learning for visual servoing control of mobile robots with visibility constraints. IEEE Trans Ind Electron 69(2):1898–1908
44.
Arents J, Greitans M (2022) Smart industrial robot control trends, challenges and opportunities within manufacturing. Appl Sci 12(2):937
45.
Cui F, Cui Q, Song Y (2021) A survey on learning-based approaches for modeling and classification of human-machine dialog systems. IEEE Trans Neural Netw Learn Syst 32(4):1418–1432
46.
Mekrache A, Bradai A, Moulay E, Dawaliby S (2022) Deep reinforcement learning techniques for vehicular networks: recent advances and future trends towards 6G. Veh Commun 33:100398
47.
Le N, Rathour VS, Yamazaki K, Luu K, Savvides M (2022) Deep reinforcement learning in computer vision: a comprehensive survey. Artif Intell Rev 55(4):2733–2819
48.
Hasselt H (2010) Double q-learning. In: Advances in neural information processing systems, December 6-9, 2010, Vancouver, British Columbia, Canada
49.
Silver D, Lever G, Heess N, Degris T, Wierstra D, Riedmiller M (2014) Deterministic policy gradient algorithms. In: International conference on machine learning, pp 387–395
50.
Correia AdS, Colombini EL (2022) Attention, please! a survey of neural attention models in deep learning. Artif Intell Rev 55:6037–6124
51.
Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I (2017) Attention is all you need. In: Advances in neural information processing systems, vol 30, 2017, December 4–9, 2017, Long Beach, CA, USA, pp 5998–6008
52.
Long Y, Xiang R, Lu Q, Huang C-R, Li M (2021) Improving attention model based on cognition grounded data for sentiment analysis. IEEE Trans Affect Comput 12(4):900–912
53.
Li X, Liu L, Tu Z, Li G, Shi S, Meng MQ-H (2021) Attending from foresight: a novel attention mechanism for neural machine translation. IEEE/ACM Trans Audio Speech Lang Process 29:2606–2616
54.
Dosovitskiy A, Beyer L, Kolesnikov A, Weissenborn D, Zhai X, Unterthiner T, Dehghani M, Minderer M, Heigold G, Gelly S, Uszkoreit J, Houlsby N (2021) An image is worth 16x16 words: transformers for image recognition at scale. In: 9th international conference on learning representations, ICLR 2021, Virtual Event, Austria, May 3–7, 2021
55.
Liang D, Chen Q, Liu Y (2021) Gated multi-attention representation in reinforcement learning. Knowl-Based Syst 233:107535
56.
Fang K, Toshev A, Fei-Fei L, Savarese S (2019) Scene memory transformer for embodied agents in long-horizon tasks. In: IEEE conference on computer vision and pattern recognition, CVPR 2019, Long Beach, CA, USA, June 16–20, pp 538–547
57.
Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: 3rd international conference on learning representations (ICLR 2015). ICLR, San Diego, CA, USA
Metadata
Title
Twin attentive deep reinforcement learning for multi-agent defensive convoy
Authors
Dongyu Fan
Haikuo Shen
Lijing Dong
Publication date
27-12-2022
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 6/2023
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-022-01759-5

Other articles of this Issue 6/2023

International Journal of Machine Learning and Cybernetics 6/2023 Go to the issue

Original Article

Robust two-phase registration method for three-dimensional point set under the Bayesian mixture framework

Original Article

Progressive image dehazing network based on dual feature extraction modules

Original Article

Context-sensitive graph representation learning

Original Article

Android malware adversarial attacks based on feature importance prediction

Correction

Correction to: robust two-phase registration method for three-dimensional point set under the bayesian mixture framework

Original Article

Enhanced feature Fusion structure of YOLO v5 for detecting small defects on metal surfaces