Toward Approximate Adaptive Learning

The problem considered in this paper is how the classification of observed behaviour of organisms can be used to influence adaptive learning, beneficially. The solution to this problem hearkens back to the pioneering work during the 1980s by Zdzisław Pawlak and others on classification of objects and approximation spaces, where elementary sets of equivalent objects a framework for perceptions concerning observed behaviours. The seminal work by Oliver Selfridge and Chris J.C.H. Watkins on delay rewards and adaptive learning, also during the 1980s, combined with more recent work on reinforcement learning provide a basis for the forms of adaptive learning introduced in this article. In addition, recent work on approximation spaces has led to what is known as approximate adaptive learning. This article presents two forms of run-and-twiddle (RT) adaptive learning, each using the Watkins’ stopping time strategy to mark the end of an episode. Twiddling amounts to adjusting what one does to achieve a better result. This becomes more apparent in approximate RT adaptive learning introduced in this article, where a record of observed behaviour patterns during each episode recorded in an ethogram makes it possible to define a pattern-based learning rate in the context of approximation spaces. Both forms of adaptive learning are actor-critic methods. The contribution of this article is the introduction of two forms of adaptive learning with Watkins’ stopping time strategy with differential discount on returns in both cases and differential learning rate for adaptive learning in the context of approximation spaces.