Skip to main content
Top

2022 | OriginalPaper | Chapter

Multi-agent Simulation for AI Behaviour Discovery in Operations Research

Authors : Michael Papasimeon, Lyndon Benke

Published in: Multi-Agent-Based Simulation XXII

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We describe ACE0, a lightweight platform for evaluating the suitability and viability of AI methods for behaviour discovery in multi-agent simulations. Specifically, ACE0 was designed to explore AI methods for multi-agent simulations used in operations research studies related to new technologies such as autonomous aircraft. Simulation environments used in production are often high-fidelity, complex, require significant domain knowledge and as a result have high R&D costs. Minimal and lightweight simulation environments can help researchers and engineers evaluate the viability of new AI technologies for behaviour discovery in a more agile and potentially cost effective manner. In this paper we describe the motivation for the development of ACE0. We provide a technical overview of the system architecture, describe a case study of behaviour discovery in the aerospace domain, and provide a qualitative evaluation of the system. The evaluation includes a brief description of collaborative research projects with academic partners, exploring different AI behaviour discovery methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
This is a simplified view of the situation, as one can consider higher order features such as turn rates and other time derivatives of the basic state space variables.
 
Literature
1.
go back to reference Austin, F., Carbone, G., Lewis, M.: Automated maneuvering decisions for air-to-air combat. In: In Proceedings of the Military Computing Conference. Anaheim, California, May 1987 Austin, F., Carbone, G., Lewis, M.: Automated maneuvering decisions for air-to-air combat. In: In Proceedings of the Military Computing Conference. Anaheim, California, May 1987
2.
go back to reference Burgin, G.H., Sidor, L.B.: Rule-Based Air Combat Simulation. Technical Report, Contractor Report 4160, National Aeronautics and Space Administration (NASA) (1988) Burgin, G.H., Sidor, L.B.: Rule-Based Air Combat Simulation. Technical Report, Contractor Report 4160, National Aeronautics and Space Administration (NASA) (1988)
5.
go back to reference Heinze, C., Papasimeon, M., Goss, S., Cross, M., Connell, R.: Simulating fighter pilots. In: Pěchouček, M., Thompson, S.G., Voos, H. (eds.) Defence Industry Applications of Autonomous Agents and Multi-Agent Systems. WSSAT, pp. 113–130. Birkhäuser Basel, Basel (2008). http://dx.doi.org/10.1007/978-3-7643-8571-2_7 Heinze, C., Papasimeon, M., Goss, S., Cross, M., Connell, R.: Simulating fighter pilots. In: Pěchouček, M., Thompson, S.G., Voos, H. (eds.) Defence Industry Applications of Autonomous Agents and Multi-Agent Systems. WSSAT, pp. 113–130. Birkhäuser Basel, Basel (2008). http://​dx.​doi.​org/​10.​1007/​978-3-7643-8571-2_​7
6.
go back to reference Hossam, M., Le, T., Huynh, V., Papasimeon, M., Phung, D.Q.: OptiGAN: generative adversarial networks for goal optimized sequence generation. In: International Joint Conference on Neural Networks (IJCNN). Glasgow, Scotland, UK, July 2020 Hossam, M., Le, T., Huynh, V., Papasimeon, M., Phung, D.Q.: OptiGAN: generative adversarial networks for goal optimized sequence generation. In: International Joint Conference on Neural Networks (IJCNN). Glasgow, Scotland, UK, July 2020
8.
go back to reference Jones, R.M., Wray, R., van Lent, M., Laird, J.E.: Planning in the Tactical Air Domain. Technical Report, aAAI Technical Report FS-94-01, AAAI (1994) Jones, R.M., Wray, R., van Lent, M., Laird, J.E.: Planning in the Tactical Air Domain. Technical Report, aAAI Technical Report FS-94-01, AAAI (1994)
10.
go back to reference Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: An empirical study of reward structures for actor-critic reinforcement learning in air combat manoeuvring simulation. In: Liu, J., Bailey, J. (eds.) AI 2019. LNCS (LNAI), vol. 11919, pp. 54–65. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-35288-2_5CrossRef Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: An empirical study of reward structures for actor-critic reinforcement learning in air combat manoeuvring simulation. In: Liu, J., Bailey, J. (eds.) AI 2019. LNCS (LNAI), vol. 11919, pp. 54–65. Springer, Cham (2019). https://​doi.​org/​10.​1007/​978-3-030-35288-2_​5CrossRef
11.
go back to reference Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: Discrete-to-Deep supervised policy learning: an effective training method for neural reinforcement learning. In: ALA 2020: Adaptive Learning Agents Workshop at AAMAS 2020. Auckland, New Zealand (2020) Kurniawan, B., Vamplew, P., Papasimeon, M., Dazeley, R., Foale, C.: Discrete-to-Deep supervised policy learning: an effective training method for neural reinforcement learning. In: ALA 2020: Adaptive Learning Agents Workshop at AAMAS 2020. Auckland, New Zealand (2020)
13.
go back to reference Lipovetzky, N., Geffner, H.: Width and serialization of classical planning problems. In: ECAI, pp. 540–545 (2012) Lipovetzky, N., Geffner, H.: Width and serialization of classical planning problems. In: ECAI, pp. 540–545 (2012)
14.
15.
go back to reference Masek, M., Lam, C.P., Benke, L., Kelly, L., Papasimeon, M.: Discovering emergent agent behaviour with evolutionary finite state machines. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds.) PRIMA 2018. LNCS (LNAI), vol. 11224, pp. 19–34. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-03098-8_2CrossRef Masek, M., Lam, C.P., Benke, L., Kelly, L., Papasimeon, M.: Discovering emergent agent behaviour with evolutionary finite state machines. In: Miller, T., Oren, N., Sakurai, Y., Noda, I., Savarimuthu, B.T.R., Cao Son, T. (eds.) PRIMA 2018. LNCS (LNAI), vol. 11224, pp. 19–34. Springer, Cham (2018). https://​doi.​org/​10.​1007/​978-3-030-03098-8_​2CrossRef
16.
go back to reference Masek, M., Lam, C.P., Kelly, L., Benke, L., Papasimeon, M.: A genetic programming framework for novel behaviour discovery in air combat scenarios. In: Ernst, A.T., Dunstall, S., García-Flores, R., Grobler, M., Marlow, D. (eds.) Data and Decision Sciences in Action 2. LNMIE, pp. 263–277. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-60135-5_19CrossRef Masek, M., Lam, C.P., Kelly, L., Benke, L., Papasimeon, M.: A genetic programming framework for novel behaviour discovery in air combat scenarios. In: Ernst, A.T., Dunstall, S., García-Flores, R., Grobler, M., Marlow, D. (eds.) Data and Decision Sciences in Action 2. LNMIE, pp. 263–277. Springer, Cham (2021). https://​doi.​org/​10.​1007/​978-3-030-60135-5_​19CrossRef
17.
go back to reference McGrew, J.S., How, J.P.: Air combat strategy using approximate dynamic programming. J. Guidance Control Dyn. 33, 1641–1654 (2010)CrossRef McGrew, J.S., How, J.P.: Air combat strategy using approximate dynamic programming. J. Guidance Control Dyn. 33, 1641–1654 (2010)CrossRef
18.
go back to reference Papasimeon, M., Pearce, A., Goss, S.: The human agent virtual environment. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. AAMAS 2007. Association for Computing Machinery, New York, NY, USA (2007). https://doi.org/10.1145/1329125.1329463 Papasimeon, M., Pearce, A., Goss, S.: The human agent virtual environment. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems. AAMAS 2007. Association for Computing Machinery, New York, NY, USA (2007). https://​doi.​org/​10.​1145/​1329125.​1329463
19.
go back to reference Park, H., Lee, B.Y., Tahk, M.J., Yoo, D.W.: Differential game based air combat maneuver generation using scoring function matrix. Int. J. Aeronaut. Space Sci. 17(2), 204–213 (2016)CrossRef Park, H., Lee, B.Y., Tahk, M.J., Yoo, D.W.: Differential game based air combat maneuver generation using scoring function matrix. Int. J. Aeronaut. Space Sci. 17(2), 204–213 (2016)CrossRef
20.
go back to reference Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9(3), 21–29 (2007)CrossRef Pérez, F., Granger, B.E.: IPython: a system for interactive scientific computing. Comput. Sci. Eng. 9(3), 21–29 (2007)CrossRef
21.
go back to reference Ramirez, M., Papasimeon, M., Benke, L., Lipovetzky, N., Miller, T., Pearce, A.R.: Real-Time UAV maneuvering via automated planning in simulations. In: 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 5243–5245. Melbourne, Australia, August 2017 Ramirez, M., Papasimeon, M., Benke, L., Lipovetzky, N., Miller, T., Pearce, A.R.: Real-Time UAV maneuvering via automated planning in simulations. In: 26th International Joint Conference on Artificial Intelligence (IJCAI), pp. 5243–5245. Melbourne, Australia, August 2017
22.
go back to reference Ramirez, M., et al.: Integrated hybrid planning and programmed control for real time UAV maneuvering. In: 17th Int. Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 1318–1326. Stockholm, Sweden, July 2018 Ramirez, M., et al.: Integrated hybrid planning and programmed control for real time UAV maneuvering. In: 17th Int. Conference on Autonomous Agents and Multiagent Systems (AAMAS), pp. 1318–1326. Stockholm, Sweden, July 2018
23.
go back to reference Shaw, R.L.: Fighter Combat: Tactics and Maneuvering. Naval Institute Press, Annapolis (1985) Shaw, R.L.: Fighter Combat: Tactics and Maneuvering. Naval Institute Press, Annapolis (1985)
24.
go back to reference Smith, R., Dike, B., Ravichandran, B., El-Fallah, A., Mehra, K.: Discovering novel fighter combat maneuvers: simulating test pilot creativity. In: Bentley, P.J., Corne, D.W. (eds.) Creative Evolutionary Systems, p. 467 - VIII. The Morgan Kaufmann Series in Artificial Intelligence, Morgan Kaufmann, San Francisco (2002). https://doi.org/10.1016/B978-155860673-9/50059-8 Smith, R., Dike, B., Ravichandran, B., El-Fallah, A., Mehra, K.: Discovering novel fighter combat maneuvers: simulating test pilot creativity. In: Bentley, P.J., Corne, D.W. (eds.) Creative Evolutionary Systems, p. 467 - VIII. The Morgan Kaufmann Series in Artificial Intelligence, Morgan Kaufmann, San Francisco (2002). https://​doi.​org/​10.​1016/​B978-155860673-9/​50059-8
25.
go back to reference Tidhar, G., Heinze, C., Selvestrel, M.: Flying together: modelling air mission teams. Appl. Intell. 8(3), 195–218 (1998)CrossRef Tidhar, G., Heinze, C., Selvestrel, M.: Flying together: modelling air mission teams. Appl. Intell. 8(3), 195–218 (1998)CrossRef
26.
go back to reference Toubman, A.: Calculated Moves: Generating Air Combat Behaviour. Ph.D. thesis, Leiden University, The Netherlands (2020) Toubman, A.: Calculated Moves: Generating Air Combat Behaviour. Ph.D. thesis, Leiden University, The Netherlands (2020)
27.
go back to reference Vered, M., Kaminka, G.A.: Heuristic online goal recognition in continuous domains. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17), pp. 4447–4454. Melbourne, Australia (2017) Vered, M., Kaminka, G.A.: Heuristic online goal recognition in continuous domains. In: Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17), pp. 4447–4454. Melbourne, Australia (2017)
Metadata
Title
Multi-agent Simulation for AI Behaviour Discovery in Operations Research
Authors
Michael Papasimeon
Lyndon Benke
Copyright Year
2022
DOI
https://doi.org/10.1007/978-3-030-94548-0_6

Premium Partner