Matthew Hausknecht
ABSTRACT
This paper considers the challenge of enabling agents to learn with as little domain-specific knowledge as possible. The main contribution is HyperNEAT-GGP, a HyperNEAT-based General Game Playing approach to Atari games. By leveraging the geometric regularities present in the Atari game screen, HyperNEAT effectively evolves policies for playing two different Atari games, Asterix and Freeway. Results show that HyperNEAT-GGP outperforms existing benchmarks on these games. HyperNEAT-GGP represents a step towards the ambitious goal of creating an agent capable of learning and seamlessly transitioning between many different tasks.
- M. Campbell, A. J. H. Jr., and F. hsiung Hsu. Deep blue. Artif. Intell., 134(1-2):57--83, 2002. Google Scholar
Digital Library
- J. Clune, B. E. Beckmann, C. Ofria, and R. T. Pennock. Evolving coordinated quadruped gaits with the hyperneat generative encoding. In Proceedings of the Eleventh conference on Congress on Evolutionary Computation, CEC'09, pages 2764--2771, Piscataway, NJ, USA, 2009. IEEE Press. Google ScholarDigital Library
- D. B. D'Ambrosio and K. O. Stanley. Generative encoding for multiagent learning. In GECCO '08: Proceedings of the 10th annual conference on Genetic and evolutionary computation, pages 819--826, New York, NY, USA, 2008. ACM. Google ScholarDigital Library
- C. Diuk, A. Cohen, and M. L. Littman. An object-oriented representation for efficient reinforcement learning. In Proceedings of 25th International Conference on Machine Learning (ICML), pages 240--247, 2008. Google ScholarDigital Library
- G. Edgers. Atari and the deep history of video games. http://www.boston.com/bostonglobe/ideas/articles/2009/03/08/a_talk_with_nick_montfort/.Google Scholar
- D. Ferrucci, E. Brown, J. Chu-Carroll, J. Fan, D. Gondek, A. A. Kalyanpur, A. Lally, J. W. Murdock, E. Nyberg, J. Prager, N. Schlaefer, and C. Welty. Building Watson: An Overview of the DeepQA Project. AI Magazine, 31(3), 2010.Google Scholar
- J. Gauci and K. O. Stanley. A case study on the critical role of geometric regularity in machine learning. In Proceedings of the 23rd National Conference on Artificial Intelligence (AAAI), 2008. Google ScholarDigital Library
- M. Genesereth and N. Love. General game playing: Overview of the aaai competition. AI Magazine, 26:62--72, 2005.Google ScholarDigital Library
- S. M. Lucas. Ms pac-man competition (screen capture mode). http://dces.essex.ac.uk/staff/sml/pacman/CIG2011Results.html.Google Scholar
- S. M. Lucas. Ms pac-man competition. SIGEVOlution, 2(4):37--38, 2007. Google ScholarDigital Library
- Y. Naddaf. Game-independent ai agents for playing atari 2600 console games. Master's thesis, University of Alberta, 2010.Google Scholar
- M. Parker and B. Bryant. Backpropagation without human supervision for visual control in quake ii. Proceedings of the 2009 IEEE Symposium on Computational Intelligence and Games (CIG'09), pages 287--293, 2009. Google ScholarDigital Library
- K. O. Stanley and R. Miikkulainen. Evolving neural networks through augmenting topologies. Evolutionary Computation, 10(2):99--127, 2002. Google ScholarDigital Library
- P. Stone and R. S. Sutton. Scaling reinforcement learning toward RoboCup soccer. In Proceedings of the Eighteenth International Conference on Machine Learning, pages 537--544. Morgan Kaufmann, San Francisco, CA, 2001. Google ScholarDigital Library
- P. Stone, R. S. Sutton, and G. Kuhlmann. Reinforcement learning for RoboCup-soccer keepaway. Adaptive Behavior, 13(3):165--188, 2005.Google ScholarCross Ref
- R. S. Sutton and A. G. Barto. Reinforcement learning: An introduction. IEEE Transactions on Neural Networks, 9(5):1054--1054, 1998. Google ScholarDigital Library
- G. Tesauro. Td-gammon, a self-teaching backgammon program, achieves master-level play. Neural Comput., 6:215--219, March 1994. Google ScholarDigital Library
- P. Verbancsics and K. O. Stanley. Evolving static representations for task transfer. J. Mach. Learn. Res., 11:1737--1769, August 2010. Google ScholarDigital Library
Index Terms
- HyperNEAT-GGP: a hyperNEAT-based atari general game player
Recommendations
Investigating whether hyperNEAT produces modular neural networks
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computationHyperNEAT represents a class of neuroevolutionary algorithms that captures some of the power of natural development with a computationally efficient high-level abstraction of development. This class of algorithms is intended to provide many of the ...
Evolving the placement and density of neurons in the hyperneat substrate
GECCO '10: Proceedings of the 12th annual conference on Genetic and evolutionary computationThe Hypercube-based NeuroEvolution of Augmenting Topologies (HyperNEAT) approach demonstrated that the pattern of weights across the connectivity of an artificial neural network (ANN) can be generated as a function of its geometry, thereby allowing ...
Enhancing es-hyperneat to evolve more complex regular neural networks
GECCO '11: Proceedings of the 13th annual conference on Genetic and evolutionary computationThe recently-introduced evolvable-substrate HyperNEAT algorithm (ES-HyperNEAT) demonstrated that the placement and density of hidden nodes in an artificial neural network can be determined based on implicit information in an infinite-resolution pattern ...
Comments