Skip to main content
Erschienen in: Artificial Life and Robotics 3/2020

30.04.2020 | Original Article

Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation

verfasst von: Takuya Fukushima, Tomoharu Nakashima, Hidehisa Akiyama

Erschienen in: Artificial Life and Robotics | Ausgabe 3/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In the RoboCup soccer simulation 2D league, players make a decision at each cycle in real time. The performance of a team highly depends on the agents’ decision-making process, which is composed of a action planning method and an evaluation function of the soccer field. In this work, a cooperative action planning based on the tree search is employed. Each action is evaluated by an evaluation function. We employ a multi-layered perceptron to construct an evaluation function. We examine the performance of the soccer agents when various sets of features are used as the input of the neural network. A feature vector is made of kick sequences executed by an expert team extracted from log files. To investigate the efficiency of our approach, we compare the performance of a team using an evaluation function modeled by neural networks against a team using a hand-tuned evaluation function.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Silver D, Hubert T, Schrittwieser J et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Nature 362(6419):1140–1144MathSciNetMATH Silver D, Hubert T, Schrittwieser J et al (2018) A general reinforcement learning algorithm that masters chess, shogi, and Go through self-play. Nature 362(6419):1140–1144MathSciNetMATH
2.
Zurück zum Zitat Kitano H, Asada M, Kuniyoshi Y et al (1997) RoboCup: a challenge problem for AI and robotics. In: RoboCup-97: Robot Soccer World Cup I, Springer, Berlin, pp 1–19 Kitano H, Asada M, Kuniyoshi Y et al (1997) RoboCup: a challenge problem for AI and robotics. In: RoboCup-97: Robot Soccer World Cup I, Springer, Berlin, pp 1–19
3.
Zurück zum Zitat Akiyama H, Aramaki S, Nakashima T (2012) Online cooperative behavior planning using a tree search method in the RoboCup soccer simulation. In: Proceedings of 4th IEEE international conference on intelligent networking and collaborative systems, pp 170–177 Akiyama H, Aramaki S, Nakashima T (2012) Online cooperative behavior planning using a tree search method in the RoboCup soccer simulation. In: Proceedings of 4th IEEE international conference on intelligent networking and collaborative systems, pp 170–177
4.
Zurück zum Zitat Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Networks 61:85–117CrossRef Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Networks 61:85–117CrossRef
5.
Zurück zum Zitat Liu W, Wang Z, Liu X et al (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26CrossRef Liu W, Wang Z, Liu X et al (2017) A survey of deep neural network architectures and their applications. Neurocomputing 234:11–26CrossRef
6.
Zurück zum Zitat Warnell G, Waytowich N, Lawhern V et al (2018) Deep TAMER: interactive agent shaping in high-dimensional state spaces. In: Proceedings of the 32nd AAAI conference on artificial intelligence (AAAI-18), pp 1545–1554 Warnell G, Waytowich N, Lawhern V et al (2018) Deep TAMER: interactive agent shaping in high-dimensional state spaces. In: Proceedings of the 32nd AAAI conference on artificial intelligence (AAAI-18), pp 1545–1554
7.
Zurück zum Zitat Stanescu M, Barriga NA, Hess A et al (2016) Evaluating real-time strategy game states using convolutional neural networks. In: Proceedings of the IEEE conference on computational intelligence and games, pp 1–7 Stanescu M, Barriga NA, Hess A et al (2016) Evaluating real-time strategy game states using convolutional neural networks. In: Proceedings of the IEEE conference on computational intelligence and games, pp 1–7
8.
Zurück zum Zitat Hong Z, Su S, Shann T et al (2018) A deep policy inference Q-network for multi-agent systems. In: Proceedings of the 17th international conference on autonomous agents and multiagent systems (AAMAS’18), pp 1388–1396 Hong Z, Su S, Shann T et al (2018) A deep policy inference Q-network for multi-agent systems. In: Proceedings of the 17th international conference on autonomous agents and multiagent systems (AAMAS’18), pp 1388–1396
9.
Zurück zum Zitat Floyd MW, Esfandiari B, Lam K (2008) A case-based reasoning approach to imitating RoboCup players. In: Proceedings of the 21st international FLAIRS conference, pp 251–256 Floyd MW, Esfandiari B, Lam K (2008) A case-based reasoning approach to imitating RoboCup players. In: Proceedings of the 21st international FLAIRS conference, pp 251–256
10.
Zurück zum Zitat Hausknecht M, Stone P (2016) Deep reinforcement learning in parameterized action space. In: Proceedings of the international conference on learning representations (ICLR), pp 1–12 Hausknecht M, Stone P (2016) Deep reinforcement learning in parameterized action space. In: Proceedings of the international conference on learning representations (ICLR), pp 1–12
11.
Zurück zum Zitat Liu Y, Stone P (2006) Value-function-based transfer for reinforcement learning using structure mapping. In: Proceedings of the 21st national conference on artificial intelligence, pp 415–420 Liu Y, Stone P (2006) Value-function-based transfer for reinforcement learning using structure mapping. In: Proceedings of the 21st national conference on artificial intelligence, pp 415–420
12.
Zurück zum Zitat Akiyama H, Nakashima T (2013) Helios base: an open source package for the robocup soccer 2D simulation. In: RoboCup 2012: Robot Soccer World Cup XVI, Springer, Berlin, pp 528–535 Akiyama H, Nakashima T (2013) Helios base: an open source package for the robocup soccer 2D simulation. In: RoboCup 2012: Robot Soccer World Cup XVI, Springer, Berlin, pp 528–535
13.
Zurück zum Zitat Sugino T, Ito T, Arimura Y et al (2018) Team HillStone2018 in the 2DSimulation league team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6 Sugino T, Ito T, Arimura Y et al (2018) Team HillStone2018 in the 2DSimulation league team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6
14.
Zurück zum Zitat Akiyama H, Nakashima T, Suzuki Y et al (2018) HELIOS2018: Team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6 Akiyama H, Nakashima T, Suzuki Y et al (2018) HELIOS2018: Team description paper. In: RoboCup2018, Montreal, Canada, pp 1–6
15.
Zurück zum Zitat Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the 14th international conference on artificial intelligence and statistics, pp 315–323 Glorot X, Bordes A, Bengio Y (2011) Deep sparse rectifier neural networks. In: Proceedings of the 14th international conference on artificial intelligence and statistics, pp 315–323
16.
Zurück zum Zitat Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference on learning representations (ICLR), pp 1–15 Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference on learning representations (ICLR), pp 1–15
Metadaten
Titel
Evaluation-function modeling with multi-layered perceptron for RoboCup soccer 2D simulation
verfasst von
Takuya Fukushima
Tomoharu Nakashima
Hidehisa Akiyama
Publikationsdatum
30.04.2020
Verlag
Springer Japan
Erschienen in
Artificial Life and Robotics / Ausgabe 3/2020
Print ISSN: 1433-5298
Elektronische ISSN: 1614-7456
DOI
https://doi.org/10.1007/s10015-020-00602-w

Weitere Artikel der Ausgabe 3/2020

Artificial Life and Robotics 3/2020 Zur Ausgabe

Neuer Inhalt