Skip to main content
Top
Published in: Neural Computing and Applications 1/2022

14-10-2021 | Review

A review of artificial intelligence applied to path planning in UAV swarms

Authors: Alejandro Puente-Castro, Daniel Rivero, Alejandro Pazos, Enrique Fernandez-Blanco

Published in: Neural Computing and Applications | Issue 1/2022

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Path Planning problems with Unmanned Aerial Vehicles (UAVs) are among the most studied knowledge areas in the related literature. However, few of them have been applied to groups of UAVs. The use of swarms allows to speed up the flight time and, thus, reducing the operational costs. When combined with Artificial Intelligence (AI) algorithms, a single system or operator can control all aircraft while optimal paths for each one can be computed. In order to introduce the current situation of these AI-based systems, a review of the most novel and relevant articles was carried out. This review was performed in two steps: first, a summary of the found articles; second, a quantitative analysis of the publications found based on different factors, such as the temporal evolution or the number of articles found based on different criteria. Therefore, this review provides not only a summary of the most recent work but it gives an overview of the trend in the use of AI algorithms in UAV swarms for Path Planning problems. The AI techniques of the articles found can be separated into four main groups based on their technique: reinforcement Learning techniques, Evolutive Computing techniques, Swarm Intelligence techniques, and, Graph Neural Networks. The final results show an increase in publications in recent years and that there is a change in the predominance of the most widely used techniques.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Akhloufi MA, Arola S, Bonnet A (2019) Drones chasing drones: reinforcement learning and deep search area proposal. Drones 3(3):58 Akhloufi MA, Arola S, Bonnet A (2019) Drones chasing drones: reinforcement learning and deep search area proposal. Drones 3(3):58
2.
go back to reference Albani D, IJsselmuiden J, Haken R, Trianni V (2017) Monitoring and mapping with robot swarms for agricultural applications. In: 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6 Albani D, IJsselmuiden J, Haken R, Trianni V (2017) Monitoring and mapping with robot swarms for agricultural applications. In: 2017 14th IEEE international conference on advanced video and signal based surveillance (AVSS). IEEE, pp 1–6
4.
go back to reference Austin R (2011) Unmanned aircraft systems: UAVS design, development and deployment, vol 54. Wiley, London Austin R (2011) Unmanned aircraft systems: UAVS design, development and deployment, vol 54. Wiley, London
5.
go back to reference Bachmann RJ, Boria FJ, Vaidyanathan R, Ifju PG, Quinn RD (2009) A biologically inspired micro-vehicle capable of aerial and terrestrial locomotion. Mech Mach Theory 44(3):513–526MATH Bachmann RJ, Boria FJ, Vaidyanathan R, Ifju PG, Quinn RD (2009) A biologically inspired micro-vehicle capable of aerial and terrestrial locomotion. Mech Mach Theory 44(3):513–526MATH
6.
go back to reference Bakker B, Zivkovic Z, Krose B (2005) Hierarchical dynamic programming for robot path planning. In: 2005 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 2756–2761 Bakker B, Zivkovic Z, Krose B (2005) Hierarchical dynamic programming for robot path planning. In: 2005 IEEE/RSJ international conference on intelligent robots and systems. IEEE, pp 2756–2761
7.
go back to reference Baldazo D, Parras J, Zazo S (2019) Decentralized multi-agent deep reinforcement learning in swarms of drones for flood monitoring. In: 2019 27th European signal processing conference (EUSIPCO). IEEE, pp 1–5 Baldazo D, Parras J, Zazo S (2019) Decentralized multi-agent deep reinforcement learning in swarms of drones for flood monitoring. In: 2019 27th European signal processing conference (EUSIPCO). IEEE, pp 1–5
8.
go back to reference Bauso D, Giarré L, Pesenti R (2004) Multiple uav cooperative path planning via neuro-dynamic programming. In: 2004 43rd IEEE conference on decision and control (CDC) (IEEE Cat. No. 04CH37601), vol 1. IEEE, pp 1087–1092 Bauso D, Giarré L, Pesenti R (2004) Multiple uav cooperative path planning via neuro-dynamic programming. In: 2004 43rd IEEE conference on decision and control (CDC) (IEEE Cat. No. 04CH37601), vol 1. IEEE, pp 1087–1092
9.
go back to reference Beni G (2004) From swarm intelligence to swarm robotics. In: International workshop on swarm robotics. Springer, pp 1–9 Beni G (2004) From swarm intelligence to swarm robotics. In: International workshop on swarm robotics. Springer, pp 1–9
10.
go back to reference Beni G, Wang J (1993) Swarm intelligence in cellular robotic systems. In: Robots and biological systems: towards a new bionics?. Springer, pp 703–712 Beni G, Wang J (1993) Swarm intelligence in cellular robotic systems. In: Robots and biological systems: towards a new bionics?. Springer, pp 703–712
11.
go back to reference Bishop CM (2006) Pattern recognition. Mach. Learn. 128(9) Bishop CM (2006) Pattern recognition. Mach. Learn. 128(9)
12.
go back to reference Bonabeau E, Meyer C (2001) Swarm intelligence: a whole new way to think about business. Harv Bus Rev 79(5):106–115 Bonabeau E, Meyer C (2001) Swarm intelligence: a whole new way to think about business. Harv Bus Rev 79(5):106–115
13.
go back to reference Buckley J (2006) Air power in the age of total war. Routledge, London Buckley J (2006) Air power in the age of total war. Routledge, London
14.
go back to reference Bürkle A, Segor F, Kollmann M (2011) Towards autonomous micro uav swarms. J Intell Robot Syst 61(1–4):339–353 Bürkle A, Segor F, Kollmann M (2011) Towards autonomous micro uav swarms. J Intell Robot Syst 61(1–4):339–353
15.
go back to reference Campion M, Ranganathan P, Faruque S (2018) A review and future directions of uav swarm communication architectures. In: 2018 IEEE international conference on electro/information technology (EIT). IEEE, pp 0903–0908 Campion M, Ranganathan P, Faruque S (2018) A review and future directions of uav swarm communication architectures. In: 2018 IEEE international conference on electro/information technology (EIT). IEEE, pp 0903–0908
16.
go back to reference Cekmez U, Ozsiginan M, Sahingoz OK (2016) Multi-uav path planning with parallel genetic algorithms on cuda architecture. In: Proceedings of the 2016 on genetic and evolutionary computation conference companion. ACM, pp 1079–1086 Cekmez U, Ozsiginan M, Sahingoz OK (2016) Multi-uav path planning with parallel genetic algorithms on cuda architecture. In: Proceedings of the 2016 on genetic and evolutionary computation conference companion. ACM, pp 1079–1086
17.
go back to reference Cekmez U, Ozsiginan M, Sahingoz OK (2017) Multi-uav path planning with multi colony ant optimization. In: International conference on intelligent systems design and applications. Springer, pp 407–417 Cekmez U, Ozsiginan M, Sahingoz OK (2017) Multi-uav path planning with multi colony ant optimization. In: International conference on intelligent systems design and applications. Springer, pp 407–417
18.
go back to reference Chen YJ, Chang DK, Zhang C (2020) Autonomous tracking using a swarm of uavs: a constrained multi-agent reinforcement learning approach. IEEE Trans Veh Technol 69(11):13702–13717 Chen YJ, Chang DK, Zhang C (2020) Autonomous tracking using a swarm of uavs: a constrained multi-agent reinforcement learning approach. IEEE Trans Veh Technol 69(11):13702–13717
19.
go back to reference Cimino MG, Lazzeri A, Vaglini G (2016) Using differential evolution to improve pheromone-based coordination of swarms of drones for collaborative target detection. In: ICPRAM, pp 605–610 Cimino MG, Lazzeri A, Vaglini G (2016) Using differential evolution to improve pheromone-based coordination of swarms of drones for collaborative target detection. In: ICPRAM, pp 605–610
20.
go back to reference Davis L (1991) Handbook of genetic algorithms Davis L (1991) Handbook of genetic algorithms
21.
go back to reference Dorigo M, Bonabeau E, Theraulaz G (2000) Ant algorithms and stigmergy. Futur Gener Comput Syst 16(8):851–871 Dorigo M, Bonabeau E, Theraulaz G (2000) Ant algorithms and stigmergy. Futur Gener Comput Syst 16(8):851–871
24.
go back to reference Duan H, Luo Q, Shi Y, Ma G (2013) Hybrid particle swarm optimization and genetic algorithm for multi-uav formation reconfiguration. IEEE Comput Intell Mag 8(3):16–27 Duan H, Luo Q, Shi Y, Ma G (2013) Hybrid particle swarm optimization and genetic algorithm for multi-uav formation reconfiguration. IEEE Comput Intell Mag 8(3):16–27
25.
go back to reference Duan F, Li X, Zhao Y (2018) Express uav swarm path planning with vnd enhanced memetic algorithm. In: Proceedings of the 2018 international conference on computing and data engineering. ACM, pp 93–97 Duan F, Li X, Zhao Y (2018) Express uav swarm path planning with vnd enhanced memetic algorithm. In: Proceedings of the 2018 international conference on computing and data engineering. ACM, pp 93–97
26.
go back to reference Duan H, Qiao P (2014) Pigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning. Int J Intell Comput Cybern 7(1):24–37MathSciNet Duan H, Qiao P (2014) Pigeon-inspired optimization: a new swarm intelligence optimizer for air robot path planning. Int J Intell Comput Cybern 7(1):24–37MathSciNet
28.
go back to reference Gaudiano P, Bonabeau E, Shargel B (2005) Evolving behaviors for a swarm of unmanned air vehicles. In: Proceedings 2005 IEEE Swarm intelligence symposium, 2005. SIS 2005. IEEE, pp 317–324 Gaudiano P, Bonabeau E, Shargel B (2005) Evolving behaviors for a swarm of unmanned air vehicles. In: Proceedings 2005 IEEE Swarm intelligence symposium, 2005. SIS 2005. IEEE, pp 317–324
29.
go back to reference Gestal Pose M (2010) Soft computing methods for practical environment solutions: techniques and studies: techniques and Studies. IGI Global, New York Gestal Pose M (2010) Soft computing methods for practical environment solutions: techniques and studies: techniques and Studies. IGI Global, New York
30.
go back to reference Giesbrecht J (2004) Global path planning for unmanned ground vehicles. Technical report. Defence Research and Development Suffield (Alberta) Giesbrecht J (2004) Global path planning for unmanned ground vehicles. Technical report. Defence Research and Development Suffield (Alberta)
31.
go back to reference Gläscher J, Daw N, Dayan P, O’Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66(4):585–595 Gläscher J, Daw N, Dayan P, O’Doherty JP (2010) States versus rewards: dissociable neural prediction error signals underlying model-based and model-free reinforcement learning. Neuron 66(4):585–595
32.
go back to reference Goh KC, Ng RB, Wong YK, Ho NJ, Chua MC (2021) Aerial filming with synchronized drones using reinforcement learning. Multimed Tools Appl 80:1–26 Goh KC, Ng RB, Wong YK, Ho NJ, Chua MC (2021) Aerial filming with synchronized drones using reinforcement learning. Multimed Tools Appl 80:1–26
33.
go back to reference Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning, Addison Wesley, reading, ma. Summary the applications of GA-genetic algorithm for dealing with some optimal calculations in economics Goldberg DE (1989) Genetic algorithms in search, optimization, and machine learning, Addison Wesley, reading, ma. Summary the applications of GA-genetic algorithm for dealing with some optimal calculations in economics
34.
go back to reference Goldberg DE (2006) Genetic algorithms. Pearson Education India, New York Goldberg DE (2006) Genetic algorithms. Pearson Education India, New York
36.
go back to reference Grassé PP (1959) La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp. la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes sociaux 6(1):41–80 Grassé PP (1959) La reconstruction du nid et les coordinations interindividuelles chezbellicositermes natalensis etcubitermes sp. la théorie de la stigmergie: Essai d’interprétation du comportement des termites constructeurs. Insectes sociaux 6(1):41–80
37.
go back to reference Hafez AT, Givigi SN, Yousefi S, Iskandarani M (2017) Multi-uav tactic switching via model predictive control and fuzzy q-learning. J Eng Sci Mil Technol 1(2):44–57 Hafez AT, Givigi SN, Yousefi S, Iskandarani M (2017) Multi-uav tactic switching via model predictive control and fuzzy q-learning. J Eng Sci Mil Technol 1(2):44–57
38.
go back to reference Hassanalian M, Khaki H, Khosravi M (2015) A new method for design of fixed wing micro air vehicle. Proc Inst Mech Eng Part G J Aerosp Eng 229(5):837–850 Hassanalian M, Khaki H, Khosravi M (2015) A new method for design of fixed wing micro air vehicle. Proc Inst Mech Eng Part G J Aerosp Eng 229(5):837–850
39.
go back to reference Hayat S, Yanmaz E, Muzaffar R (2016) Survey on unmanned aerial vehicle networks for civil applications: a communications viewpoint. IEEE Commun Surv Tutor 18(4):2624–2661 Hayat S, Yanmaz E, Muzaffar R (2016) Survey on unmanned aerial vehicle networks for civil applications: a communications viewpoint. IEEE Commun Surv Tutor 18(4):2624–2661
40.
go back to reference He W, Qi X, Liu L (2021) A novel hybrid particle swarm optimization for multi-uav cooperate path planning. Appl Intell 2021:1–15 He W, Qi X, Liu L (2021) A novel hybrid particle swarm optimization for multi-uav cooperate path planning. Appl Intell 2021:1–15
41.
go back to reference Hoang VT, Phung MD, Dinh TH, Zhu Q, Ha Q (2019). Reconfigurable multi-uav formation using angle-encoded pso. In: 2019 IEEE 15th international conference on automation science and engineering (CASE). IEEE, pp 1670–1675 Hoang VT, Phung MD, Dinh TH, Zhu Q, Ha Q (2019). Reconfigurable multi-uav formation using angle-encoded pso. In: 2019 IEEE 15th international conference on automation science and engineering (CASE). IEEE, pp 1670–1675
42.
go back to reference Howard LM, D’Angelo DJ (1995) The ga-p: a genetic algorithm and genetic programming hybrid. IEEE Expert 10(3):11–15 Howard LM, D’Angelo DJ (1995) The ga-p: a genetic algorithm and genetic programming hybrid. IEEE Expert 10(3):11–15
43.
go back to reference Huang T, Wang Y, Cao X, Xu D (2020). Multi-uav mission planning method. In: 2020 3rd international conference on unmanned systems (ICUS). IEEE, pp 325–330 Huang T, Wang Y, Cao X, Xu D (2020). Multi-uav mission planning method. In: 2020 3rd international conference on unmanned systems (ICUS). IEEE, pp 325–330
44.
go back to reference Hung SM, Givigi SN (2016) A q-learning approach to flocking with uavs in a stochastic environment. IEEE Trans Cybern 47(1):186–197 Hung SM, Givigi SN (2016) A q-learning approach to flocking with uavs in a stochastic environment. IEEE Trans Cybern 47(1):186–197
45.
go back to reference Hüttenrauch M, Adrian S, Neumann G et al (2019) Deep reinforcement learning for swarm systems. J Mach Learn Res 20(54):1–31MathSciNetMATH Hüttenrauch M, Adrian S, Neumann G et al (2019) Deep reinforcement learning for swarm systems. J Mach Learn Res 20(54):1–31MathSciNetMATH
47.
go back to reference Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285 Kaelbling LP, Littman ML, Moore AW (1996) Reinforcement learning: a survey. J Artif Intell Res 4:237–285
48.
go back to reference Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-international conference on neural networks, vol 4. IEEE, pp 1942–1948 Kennedy J, Eberhart R (1995) Particle swarm optimization. In: Proceedings of ICNN’95-international conference on neural networks, vol 4. IEEE, pp 1942–1948
49.
go back to reference Khalil AA, Byrne AJ, Rahman MA, Manshaei MH (2021) Efficient uav trajectory-planning using economic reinforcement learning. arXiv:2103.02676 Khalil AA, Byrne AJ, Rahman MA, Manshaei MH (2021) Efficient uav trajectory-planning using economic reinforcement learning. arXiv:​2103.​02676
50.
go back to reference Koza JR, Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection, vol 1. MIT press, New YorkMATH Koza JR, Koza JR (1992) Genetic programming: on the programming of computers by means of natural selection, vol 1. MIT press, New YorkMATH
52.
go back to reference Koziel S, Michalewicz Z (1998) A decoder-based evolutionary algorithm for constrained parameter optimization problems. In: International conference on parallel problem solving from nature. Springer, pp 231–240 Koziel S, Michalewicz Z (1998) A decoder-based evolutionary algorithm for constrained parameter optimization problems. In: International conference on parallel problem solving from nature. Springer, pp 231–240
53.
go back to reference Lamont GB, Slear JN, Melendez K (2007) Uav swarm mission planning and routing using multi-objective evolutionary algorithms. In: 2007 IEEE symposium on computational intelligence in multi-criteria decision-making, IEEE, pp 10–20 Lamont GB, Slear JN, Melendez K (2007) Uav swarm mission planning and routing using multi-objective evolutionary algorithms. In: 2007 IEEE symposium on computational intelligence in multi-criteria decision-making, IEEE, pp 10–20
54.
go back to reference LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444 LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444
55.
56.
go back to reference Li J, Sun XX (2008) A route planning’s method for unmanned aerial vehicles based on improved a-star algorithm. Acta Armamentarii 7:788–792 Li J, Sun XX (2008) A route planning’s method for unmanned aerial vehicles based on improved a-star algorithm. Acta Armamentarii 7:788–792
57.
go back to reference Liu Y, Passino KM (2000) Swarm intelligence: literature overview. Department of Electrical Engineering, The Ohio State University, Ohio Liu Y, Passino KM (2000) Swarm intelligence: literature overview. Department of Electrical Engineering, The Ohio State University, Ohio
58.
go back to reference Liu W, Zheng Z, Cai K (2013) Adaptive path planning for unmanned aerial vehicles based on bi-level programming and variable planning time interval. Chin J Aeronaut 26(3):646–660 Liu W, Zheng Z, Cai K (2013) Adaptive path planning for unmanned aerial vehicles based on bi-level programming and variable planning time interval. Chin J Aeronaut 26(3):646–660
59.
go back to reference Liu W, Zheng Z, Cai KY (2013) Bi-level programming based real-time path planning for unmanned aerial vehicles. Knowl Based Syst 44:34–47 Liu W, Zheng Z, Cai KY (2013) Bi-level programming based real-time path planning for unmanned aerial vehicles. Knowl Based Syst 44:34–47
60.
go back to reference Liu J, Wang W, Wang T, Shu Z, Li X (2018) A motif-based rescue mission planning method for uav swarms usingan improved picea. IEEE Access 6:40778–40791 Liu J, Wang W, Wang T, Shu Z, Li X (2018) A motif-based rescue mission planning method for uav swarms usingan improved picea. IEEE Access 6:40778–40791
61.
go back to reference Liu C, Xie W, Zhang P, Guo Q, Ding D (2019) Multi-uavs cooperative coverage reconnaissance with neural network and genetic algorithm. In: Proceedings of the 2019 3rd high performance computing and cluster technologies conference. ACM, pp 81–86 Liu C, Xie W, Zhang P, Guo Q, Ding D (2019) Multi-uavs cooperative coverage reconnaissance with neural network and genetic algorithm. In: Proceedings of the 2019 3rd high performance computing and cluster technologies conference. ACM, pp 81–86
62.
go back to reference Li X, Zhao Y, Zhang J, Dong Y (2016) A hybrid pso algorithm based flight path optimization for multiple agricultural uavs. In: 2016 IEEE 28th international conference on tools with artificial intelligence (ICTAI). IEEE, pp 691–697 Li X, Zhao Y, Zhang J, Dong Y (2016) A hybrid pso algorithm based flight path optimization for multiple agricultural uavs. In: 2016 IEEE 28th international conference on tools with artificial intelligence (ICTAI). IEEE, pp 691–697
63.
go back to reference Luo W, Tang Q, Fu C, Eberhard P (2018) Deep-sarsa based multi-uav path planning and obstacle avoidance in a dynamic environment. In: International conference on sensing and imaging. Springer, pp 102–111 Luo W, Tang Q, Fu C, Eberhard P (2018) Deep-sarsa based multi-uav path planning and obstacle avoidance in a dynamic environment. In: International conference on sensing and imaging. Springer, pp 102–111
64.
go back to reference Majd A, Ashraf A, Troubitsyna E, Daneshtalab M (2018). Integrating learning, optimization, and prediction for efficient navigation of swarms of drones. In: 2018 26th Euromicro international conference on parallel, distributed and network-based processing (PDP). IEEE, pp 101–108 Majd A, Ashraf A, Troubitsyna E, Daneshtalab M (2018). Integrating learning, optimization, and prediction for efficient navigation of swarms of drones. In: 2018 26th Euromicro international conference on parallel, distributed and network-based processing (PDP). IEEE, pp 101–108
66.
go back to reference Michie D, Spiegelhalter DJ, Taylor C et al (1994) Machine learning, neural and statistical classification. Citeseer 13 Michie D, Spiegelhalter DJ, Taylor C et al (1994) Machine learning, neural and statistical classification. Citeseer 13
67.
go back to reference Miller PM (2006) Mini, micro, and swarming unmanned aerial vehicles: a baseline study. Inn: Library of congress Washington DC, Federal Research Div Miller PM (2006) Mini, micro, and swarming unmanned aerial vehicles: a baseline study. Inn: Library of congress Washington DC, Federal Research Div
68.
go back to reference Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529 Mnih V, Kavukcuoglu K, Silver D, Rusu AA, Veness J, Bellemare MG, Graves A, Riedmiller M, Fidjeland AK, Ostrovski G et al (2015) Human-level control through deep reinforcement learning. Nature 518(7540):529
69.
go back to reference Moeller M, Pohl D, Gurdan T (2019) Unmanned aerial vehicle swarm photography. US Patent App. 15/811,726 Moeller M, Pohl D, Gurdan T (2019) Unmanned aerial vehicle swarm photography. US Patent App. 15/811,726
70.
go back to reference Olson JM, Bidstrup CC, Anderson BK, Parkinson AR, McLain TW (2020). Optimal multi-agent coverage and flight time with genetic path planning. In: 2020 International conference on unmanned aircraft systems (ICUAS). IEEE, pp 228–237 Olson JM, Bidstrup CC, Anderson BK, Parkinson AR, McLain TW (2020). Optimal multi-agent coverage and flight time with genetic path planning. In: 2020 International conference on unmanned aircraft systems (ICUAS). IEEE, pp 228–237
71.
go back to reference Pan Y, Yang Y, Li W (2021) A deep learning trained by genetic algorithm to improve the efficiency of path planning for data collection with multi-uav. IEEE Access 9:7994–8005 Pan Y, Yang Y, Li W (2021) A deep learning trained by genetic algorithm to improve the efficiency of path planning for data collection with multi-uav. IEEE Access 9:7994–8005
72.
go back to reference Parunak HV, Purcell M, O’Connell R (2002) Digital pheromones for autonomous coordination of swarming uav’s. In: 1st UAV conference, p 3446 Parunak HV, Purcell M, O’Connell R (2002) Digital pheromones for autonomous coordination of swarming uav’s. In: 1st UAV conference, p 3446
73.
go back to reference Payton D, Daily M, Estowski R, Howard M, Lee C (2001) Pheromone robotics. Auton Robot 11(3):319–324MATH Payton D, Daily M, Estowski R, Howard M, Lee C (2001) Pheromone robotics. Auton Robot 11(3):319–324MATH
74.
go back to reference Perez-Carabaza S, Besada-Portas E, Lopez-Orozco JA, Jesus M (2018) Ant colony optimization for multi-uav minimum time search in uncertain domains. Appl Soft Comput 62:789–806 Perez-Carabaza S, Besada-Portas E, Lopez-Orozco JA, Jesus M (2018) Ant colony optimization for multi-uav minimum time search in uncertain domains. Appl Soft Comput 62:789–806
75.
go back to reference Ramirez-Atencia C, Bello-Orgaz G, R-Moreno MD, Camacho D (2017) Solving complex multi-uav mission planning problems using multi-objective genetic algorithms. Soft Comput 21(17):4883–4900 Ramirez-Atencia C, Bello-Orgaz G, R-Moreno MD, Camacho D (2017) Solving complex multi-uav mission planning problems using multi-objective genetic algorithms. Soft Comput 21(17):4883–4900
76.
go back to reference Ramirez-Atencia C, R-Moreno MD, Camacho D (2017) Handling swarm of uavs based on evolutionary multi-objective optimization. Progr Artif Intell 6(3):263–274 Ramirez-Atencia C, R-Moreno MD, Camacho D (2017) Handling swarm of uavs based on evolutionary multi-objective optimization. Progr Artif Intell 6(3):263–274
77.
go back to reference Rosenblatt F (1961) Principles of neurodynamics. Perceptrons and the theory of brain mechanisms. Technical report, Cornell Aeronautical Lab Inc, Buffalo Rosenblatt F (1961) Principles of neurodynamics. Perceptrons and the theory of brain mechanisms. Technical report, Cornell Aeronautical Lab Inc, Buffalo
78.
go back to reference Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386 Rosenblatt F (1958) The perceptron: a probabilistic model for information storage and organization in the brain. Psychol Rev 65(6):386
79.
go back to reference Roudneshin M, Sizkouhi AMM, Aghdam AG (2019) Effective learning algorithms for search and rescue missions in unknown environments. In: 2019 IEEE international conference on wireless for space and extreme environments (WiSEE). IEEE, pp 76–80 Roudneshin M, Sizkouhi AMM, Aghdam AG (2019) Effective learning algorithms for search and rescue missions in unknown environments. In: 2019 IEEE international conference on wireless for space and extreme environments (WiSEE). IEEE, pp 76–80
80.
go back to reference Roy S, Biswas S, Chaudhuri SS (2014) Nature-inspired swarm intelligence and its applications. Int J Modern Educ Comput Sci 6(12):55 Roy S, Biswas S, Chaudhuri SS (2014) Nature-inspired swarm intelligence and its applications. Int J Modern Educ Comput Sci 6(12):55
81.
go back to reference Rui P (2010) Multi-uav formation maneuvering control based on q-learning fuzzy controller. In: 2nd international conference on advanced computer control, vol 4. IEEE, pp 252–257 Rui P (2010) Multi-uav formation maneuvering control based on q-learning fuzzy controller. In: 2nd international conference on advanced computer control, vol 4. IEEE, pp 252–257
82.
go back to reference Rummery GA, Niranjan M (1994) On-line Q-learning using connectionist systems, vol 37. Department of Engineering Cambridge, University of Cambridge, London Rummery GA, Niranjan M (1994) On-line Q-learning using connectionist systems, vol 37. Department of Engineering Cambridge, University of Cambridge, London
83.
go back to reference Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, MalaysiaMATH Russell SJ, Norvig P (2016) Artificial intelligence: a modern approach. Pearson Education Limited, MalaysiaMATH
84.
go back to reference Sahin E, Winfield AF (2008) Special issue on swarm robotics. Swarm Intell 2(2–4):69–72 Sahin E, Winfield AF (2008) Special issue on swarm robotics. Swarm Intell 2(2–4):69–72
85.
go back to reference San KT, Lee EY, Chang YS (2016). The delivery assignment solution for swarms of uavs dealing with multi-dimensional chromosome representation of genetic algorithm. In: 2016 IEEE 7th annual ubiquitous computing, electronics and mobile communication conference (UEMCON). IEEE, pp 1–7 San KT, Lee EY, Chang YS (2016). The delivery assignment solution for swarms of uavs dealing with multi-dimensional chromosome representation of genetic algorithm. In: 2016 IEEE 7th annual ubiquitous computing, electronics and mobile communication conference (UEMCON). IEEE, pp 1–7
86.
go back to reference Sathyan A, Ernest ND, Cohen K (2016) An efficient genetic fuzzy approach to uav swarm routing. Unmanned Syst 4(02):117–127 Sathyan A, Ernest ND, Cohen K (2016) An efficient genetic fuzzy approach to uav swarm routing. Unmanned Syst 4(02):117–127
87.
go back to reference Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80 Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80
89.
go back to reference Shah S, Dey D, Lovett C, Kapoor A (2018) Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and service robotics. Springer, pp 621–635 Shah S, Dey D, Lovett C, Kapoor A (2018) Airsim: high-fidelity visual and physical simulation for autonomous vehicles. In: Field and service robotics. Springer, pp 621–635
90.
go back to reference Shao S, Peng Y, He C, Du Y (2020) Efficient path planning for uav formation via comprehensively improved particle swarm optimization. ISA Trans 97:415–430 Shao S, Peng Y, He C, Du Y (2020) Efficient path planning for uav formation via comprehensively improved particle swarm optimization. ISA Trans 97:415–430
91.
go back to reference Sharkey AJ, Sharkey N (2006) The application of swarm intelligence to collective robots. In: Advances in applied artificial intelligence. IGI Global, pp 157–185 Sharkey AJ, Sharkey N (2006) The application of swarm intelligence to collective robots. In: Advances in applied artificial intelligence. IGI Global, pp 157–185
92.
go back to reference Sivanandam S, Deepa S (2008) Genetic algorithms. Introduction to genetic algorithms. Springer, pp 15–37 Sivanandam S, Deepa S (2008) Genetic algorithms. Introduction to genetic algorithms. Springer, pp 15–37
93.
go back to reference Speck C, Bucci DJ (2018). Distributed uav swarm formation control via object-focused, multi-objective sarsa. In: 2018 Annual American control conference (ACC). IEEE, pp 6596–6601 Speck C, Bucci DJ (2018). Distributed uav swarm formation control via object-focused, multi-objective sarsa. In: 2018 Annual American control conference (ACC). IEEE, pp 6596–6601
94.
go back to reference Srinivas M, Patnaik LM (1994) Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Trans Syst Man Cybern 24(4):656–667 Srinivas M, Patnaik LM (1994) Adaptive probabilities of crossover and mutation in genetic algorithms. IEEE Trans Syst Man Cybern 24(4):656–667
95.
go back to reference Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359MathSciNetMATH Storn R, Price K (1997) Differential evolution-a simple and efficient heuristic for global optimization over continuous spaces. J Global Optim 11(4):341–359MathSciNetMATH
96.
go back to reference Su Xh, Zhao M, Zhao Ll, Zhang Yh (2016) A novel multi stage cooperative path re-planning method for multi uav. In: Pacific rim international conference on artificial intelligence. Springer, pp 482–495 Su Xh, Zhao M, Zhao Ll, Zhang Yh (2016) A novel multi stage cooperative path re-planning method for multi uav. In: Pacific rim international conference on artificial intelligence. Springer, pp 482–495
97.
go back to reference Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press, New YorkMATH Sutton RS, Barto AG (2018) Reinforcement learning: an introduction. MIT press, New YorkMATH
98.
go back to reference Sutton RS, Precup D, Singh SP (1998) Intra-option learning about temporally abstract actions. ICML 98:556–564 Sutton RS, Precup D, Singh SP (1998) Intra-option learning about temporally abstract actions. ICML 98:556–564
99.
go back to reference Tan Y, Zheng Z (2013) Research advance in swarm robotics. Defence Technol 9(1):18–39 Tan Y, Zheng Z (2013) Research advance in swarm robotics. Defence Technol 9(1):18–39
100.
go back to reference Theraulaz G, Bonabeau E (1999) A brief history of stigmergy. Artif Life 5(2):97–116 Theraulaz G, Bonabeau E (1999) A brief history of stigmergy. Artif Life 5(2):97–116
101.
go back to reference Tolstaya E, Gama F, Paulos J, Pappas G, Kumar V, Ribeiro A (2020) Learning decentralized controllers for robot swarms with graph neural networks. In: Conference on robot learning. PMLR, pp 671–682 Tolstaya E, Gama F, Paulos J, Pappas G, Kumar V, Ribeiro A (2020) Learning decentralized controllers for robot swarms with graph neural networks. In: Conference on robot learning. PMLR, pp 671–682
102.
go back to reference Tseng FH, Liang TT, Lee CH, Der Chou L, Chao HC (2014) A star search algorithm for civil uav path planning with 3g communication. In: 2014 Tenth international conference on intelligent information hiding and multimedia signal processing. IEEE, pp 942–945 Tseng FH, Liang TT, Lee CH, Der Chou L, Chao HC (2014) A star search algorithm for civil uav path planning with 3g communication. In: 2014 Tenth international conference on intelligent information hiding and multimedia signal processing. IEEE, pp 942–945
103.
go back to reference Van Hasselt H, Wiering MA (2007) Reinforcement learning in continuous action spaces. In: 2007 IEEE international symposium on approximate dynamic programming and reinforcement learning. IEEE, pp 272–279 Van Hasselt H, Wiering MA (2007) Reinforcement learning in continuous action spaces. In: 2007 IEEE international symposium on approximate dynamic programming and reinforcement learning. IEEE, pp 272–279
104.
go back to reference Venturini F, Mason F, Pase F, Chiariotti F, Testolin A, Zanella A, Zorzi M (2020)Distributed reinforcement learning for flexible uav swarm control with transfer learning capabilities. In: Proceedings of the 6th ACM workshop on micro aerial vehicle networks, systems, and applications, pp 1–6 Venturini F, Mason F, Pase F, Chiariotti F, Testolin A, Zanella A, Zorzi M (2020)Distributed reinforcement learning for flexible uav swarm control with transfer learning capabilities. In: Proceedings of the 6th ACM workshop on micro aerial vehicle networks, systems, and applications, pp 1–6
105.
go back to reference Venturini F, Mason F, Pase F, Chiariotti F, Testolin A, Zanella A, Zorzi M (2021) Distributed reinforcement learning for flexible and efficient uav swarm control. arXiv:2103.04666 Venturini F, Mason F, Pase F, Chiariotti F, Testolin A, Zanella A, Zorzi M (2021) Distributed reinforcement learning for flexible and efficient uav swarm control. arXiv:​2103.​04666
106.
go back to reference Vijayakumari DM, Kim S, Suk J, Mo H (2019) Receding-horizon trajectory planning for multiple uavs using particle swarm optimization. In: AIAA Scitech 2019 forum, p 1165 Vijayakumari DM, Kim S, Suk J, Mo H (2019) Receding-horizon trajectory planning for multiple uavs using particle swarm optimization. In: AIAA Scitech 2019 forum, p 1165
107.
go back to reference Wang BH, Wang DB, Ali ZA (2020) A cauchy mutant pigeon-inspired optimization-based multi-unmanned aerial vehicle path planning method. Meas Control 53(1–2):83–92 Wang BH, Wang DB, Ali ZA (2020) A cauchy mutant pigeon-inspired optimization-based multi-unmanned aerial vehicle path planning method. Meas Control 53(1–2):83–92
108.
go back to reference Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292MATH Watkins CJ, Dayan P (1992) Q-learning. Mach Learn 8(3–4):279–292MATH
111.
go back to reference Wiering M, Van Otterlo M (2012) Reinforcement learning. Adapt Learn Optim 12:3 Wiering M, Van Otterlo M (2012) Reinforcement learning. Adapt Learn Optim 12:3
113.
go back to reference Yang T, Yi X, Wu J, Yuan Y, Wu D, Meng Z, Hong Y, Wang H, Lin Z, Johansson KH (2019) A survey of distributed optimization. Annu Rev Control 47:278–305MathSciNet Yang T, Yi X, Wu J, Yuan Y, Wu D, Meng Z, Hong Y, Wang H, Lin Z, Johansson KH (2019) A survey of distributed optimization. Annu Rev Control 47:278–305MathSciNet
114.
go back to reference Yang Q, Jang SJ, Yoo SJ (2020) Q-learning-based fuzzy logic for multi-objective routing algorithm in flying ad hoc networks. Wirel Person Commun 113:1–24 Yang Q, Jang SJ, Yoo SJ (2020) Q-learning-based fuzzy logic for multi-objective routing algorithm in flying ad hoc networks. Wirel Person Commun 113:1–24
115.
go back to reference Ye F, Chen J, Tian Y, Jiang T (2020) Cooperative multiple task assignment of heterogeneous uavs using a modified genetic algorithm with multi-type-gene chromosome encoding strategy. J Intell Robot Syst 100:615–627 Ye F, Chen J, Tian Y, Jiang T (2020) Cooperative multiple task assignment of heterogeneous uavs using a modified genetic algorithm with multi-type-gene chromosome encoding strategy. J Intell Robot Syst 100:615–627
116.
go back to reference Yijing Z, Zheng Z, Xiaoyi Z, Yang L (2017). Q learning algorithm based uav path learning and obstacle avoidence approach. In: 2017 36th Chinese control conference (CCC). IEEE, pp 3397–3402 Yijing Z, Zheng Z, Xiaoyi Z, Yang L (2017). Q learning algorithm based uav path learning and obstacle avoidence approach. In: 2017 36th Chinese control conference (CCC). IEEE, pp 3397–3402
117.
go back to reference Zhang X, Ali M (2020) A bean optimization-based cooperation method for target searching by swarm uavs in unknown environments. IEEE Access 8:43850–43862 Zhang X, Ali M (2020) A bean optimization-based cooperation method for target searching by swarm uavs in unknown environments. IEEE Access 8:43850–43862
118.
go back to reference Zhao Y, Zheng Z, Liu Y (2018) Survey on computational-intelligence-based uav path planning. Knowl Based Syst 158:54–64 Zhao Y, Zheng Z, Liu Y (2018) Survey on computational-intelligence-based uav path planning. Knowl Based Syst 158:54–64
119.
go back to reference Zhao W, Fang Z, Yang Z (2020) Four-dimensional trajectory generation for uavs based on multi-agent q learning. J Navig 73(4):874–891 Zhao W, Fang Z, Yang Z (2020) Four-dimensional trajectory generation for uavs based on multi-agent q learning. J Navig 73(4):874–891
120.
go back to reference Zhao H, Pei Z, Jiang J, Guan R, Wang C, Shi X (2010) A hybrid swarm intelligent method based on genetic algorithm and artificial bee colony. In: International conference in swarm intelligence. Springer, pp 558–565 Zhao H, Pei Z, Jiang J, Guan R, Wang C, Shi X (2010) A hybrid swarm intelligent method based on genetic algorithm and artificial bee colony. In: International conference in swarm intelligence. Springer, pp 558–565
121.
go back to reference Zhao W, Qiu W, Zhou T, Shao X, Wang X (2019). Sarsa-based trajectory planning of multi-uavs in dense mesh router networks. In: 2019 international conference on wireless and mobile computing, networking and communications (WiMob). IEEE, pp 1–5 Zhao W, Qiu W, Zhou T, Shao X, Wang X (2019). Sarsa-based trajectory planning of multi-uavs in dense mesh router networks. In: 2019 international conference on wireless and mobile computing, networking and communications (WiMob). IEEE, pp 1–5
122.
go back to reference Zhao D, Wang H, Shao K, Zhu Y (2016). Deep reinforcement learning with experience replay based on sarsa. In: 2016 IEEE symposium series on computational intelligence (SSCI). IEEE, pp 1–6 Zhao D, Wang H, Shao K, Zhu Y (2016). Deep reinforcement learning with experience replay based on sarsa. In: 2016 IEEE symposium series on computational intelligence (SSCI). IEEE, pp 1–6
123.
go back to reference Zhen Z, Xing D, Gao C (2018) Cooperative search-attack mission planning for multi-uav based on intelligent self-organized algorithm. Aerosp Sci Technol 76:402–411 Zhen Z, Xing D, Gao C (2018) Cooperative search-attack mission planning for multi-uav based on intelligent self-organized algorithm. Aerosp Sci Technol 76:402–411
124.
go back to reference Zhen Z, Chen Y, Wen L, Han B (2020) An intelligent cooperative mission planning scheme of uav swarm in uncertain dynamic environment. Aerosp Sci Technol 100:105826 Zhen Z, Chen Y, Wen L, Han B (2020) An intelligent cooperative mission planning scheme of uav swarm in uncertain dynamic environment. Aerosp Sci Technol 100:105826
125.
go back to reference Zhou Z, Luo D, Shao J, Xu Y, You Y (2020) Immune genetic algorithm based multi-uav cooperative target search with event-triggered mechanism. Phys Commun 41:101103 Zhou Z, Luo D, Shao J, Xu Y, You Y (2020) Immune genetic algorithm based multi-uav cooperative target search with event-triggered mechanism. Phys Commun 41:101103
126.
go back to reference Zurada JM (1992) Introduction to artificial neural systems, vol 8. West publishing company St. Paul, Berlin Zurada JM (1992) Introduction to artificial neural systems, vol 8. West publishing company St. Paul, Berlin
Metadata
Title
A review of artificial intelligence applied to path planning in UAV swarms
Authors
Alejandro Puente-Castro
Daniel Rivero
Alejandro Pazos
Enrique Fernandez-Blanco
Publication date
14-10-2021
Publisher
Springer London
Published in
Neural Computing and Applications / Issue 1/2022
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-06569-4

Other articles of this Issue 1/2022

Neural Computing and Applications 1/2022 Go to the issue

Premium Partner