Skip to main content
Top

2018 | OriginalPaper | Chapter

Flood-Fill Q-Learning Updates for Learning Redundant Policies in Order to Interact with a Computer Screen by Clicking

Authors : Nathaniel du Preez-Wilkinson, Marcus Gallagher, Xuelei Hu

Published in: AI 2018: Advances in Artificial Intelligence

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

We present a specialisation of Q-learning for the problem of training an agent to click on a computer screen. In this problem formulation the agent sees the pixels of the screen as input, and selects a pixel as output. The task of selecting a pixel to click on involves selecting an action from a large discrete action space in which many of the actions are completely equivalent in terms of reinforcement learning state transitions. We propose to exploit this by performing simultaneous Q-learning updates for equivalent actions. We use the flood fill algorithm on the input image to determine the action (pixel) equivalence.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Footnotes
1
For the problem described in this paper all pixels of the same colour are also part of the same shape. Thus, for efficiency reasons, the flood fill algorithm was not actually implemented in this case. Instead we used the simpler method of updating the Q-values of all pixels with same colour.
 
Literature
1.
go back to reference Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998) Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction, vol. 1. MIT Press, Cambridge (1998)
2.
go back to reference Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRef Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)CrossRef
3.
go back to reference Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)CrossRef Silver, D., et al.: Mastering the game of Go with deep neural networks and tree search. Nature 529(7587), 484–489 (2016)CrossRef
4.
go back to reference Pazis,J., Parr, R.: Generalized value functions for large action sets. In: ICML, pp. 1185–1192 (2011) Pazis,J., Parr, R.: Generalized value functions for large action sets. In: ICML, pp. 1185–1192 (2011)
5.
go back to reference Shi, T., et al.: World of bits: an open-domain platform for web-based agents. In: ICML (2017) Shi, T., et al.: World of bits: an open-domain platform for web-based agents. In: ICML (2017)
6.
go back to reference Haarnoja, T., et al.: Reinforcement learning with deep energy-based policies. In: ICML (2017) Haarnoja, T., et al.: Reinforcement learning with deep energy-based policies. In: ICML (2017)
7.
go back to reference Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: AAAI (2016) Van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double Q-learning. In: AAAI (2016)
Metadata
Title
Flood-Fill Q-Learning Updates for Learning Redundant Policies in Order to Interact with a Computer Screen by Clicking
Authors
Nathaniel du Preez-Wilkinson
Marcus Gallagher
Xuelei Hu
Copyright Year
2018
DOI
https://doi.org/10.1007/978-3-030-03991-2_49

Premium Partner