Abstract
An associative memory system is presented which does not require a “teacher” to provide the desired associations. For each input key it conducts a search for the output pattern which optimizes an external payoff or reinforcement signal. The associative search network (ASN) combines pattern recognition and function optimization capabilities in a simple and effective way. We define the associative search problem, discuss conditions under which the associative search network is capable of solving it, and present results from computer simulations. The synthesis of sensory-motor control surfaces is discussed as an example of the associative search problem.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albus, J.S.: Mechanisms of planning and problem solving in the brain. Math. Biosci. 45, 247–293 (1979)
Amari, S.: Neural theory of association and concept-formation. Biol. Cybern. 27, 175–185 (1977)
Anderson, J.A., Silverstein, J.W., Ritz, S.A., Jones, R.S.: Distinctive features, categorical perception, and probability learning. Some applications of a neural model. Psychol. Rev. 85, 413–451 (1977)
Cooper, L.N.: A possible organization of animal memory and learning. In: Proceedings of the Nobel Symposium on Collective Properties of Physical Systems. Lundquist, B., Lundquist, S. (eds.) New York: Academic Press 1974
Didday, R.L.: A model of visuomotor mechanisms in the frog optic tectum. Math. Biosci, 30, 169–180 (1976)
Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. New York: Wiley 1973
Freeman, W.J.: Mass action in the nervous system. New York: Academic Press 1975
Grossberg, S.: Adaptive pattern classification and universal recoding. II. Feedback, expectation, olfaction, illusions. Biol. Cybern. 23, 187–202 (1976)
Harth, E., Tzanakou, E.: ALOPEX: a stochastic method for determining visual receptive fields. Vision Res. 14, 1475–1482 (1974)
John, E.R., Schwartz, E.L.: The neurophysiology of information processing and cognition. Annu. Rev. of Psychol. 29, 1–29 (1978)
Kasyap, R.L., Blaydon, C.C., Fu, K.S.: Stochastic approximation. In: Adaptive, learning, and pattern recognition systems: theory and applications. pp. 339–354. Mendel, J.M., Fu, K.S. (eds.). New York: Academic Press 1970
Klopf, A.H.: Brain function and adaptive systems — a heterostatic theory. Air Force Cambridge Research Laboratories research report AFCRL-72-0164, Bedford, MA. (1972) (AD742259). (A summary in: Proceedings of the International Conference on Systems, Man and Cybernetics, IEEE Systems, Man and Cybernetics Society, Dallas, Texas, 1974)
Klopf, A.H.: Goal-seeking systems from goal-sceking components: implications for AI. The Cognition and Brain Theory Newsletter, Vol. III, No. 2 (1979)
Klopf, A.H.: The hedonistic neuron: A theory of memory, learning and intelligence. Washington, D.C.: Hemisphere 1981 (to be published)
Kohonen, T.: Associative memory: a system theoretic approach. Berlin, Heidelberg, New York: Springer 1977
Kohonen, T., Oja, E.: Fast adaptive formation of orthogonalizing filters and associative memory in recurrent networks of neuronlike elements. Biol. Cybern. 21, 85–95 (1976)
Mendel, J.M., McLaren, R.W.: Reinforcement-learning control and pattern recognition systems. In: Adaptive, learning, and pattern recognition systems: theory and applications, pp. 287–317 Mendel, J.M., Fu, K.S. (eds.). New York: Academic Press 1970
Minsky, M.L., Papert, S.: Perceptron: an introduction to computational geometry. Cambridge, MA: MIT Press 1969
Nakano, K.: Associatron — a model of associative memory. IEEE Trans. Syst. Man Cybern. 3, 380–388 (1972)
Narendra, K.S., Thathachar, M.A.L.: Learning automata — a survey. IEEE Trans. Syst. Man Cybern 4, 323–334 (1974)
Nilsson, N.J.: Learning machines. New York: McGraw-Hill 1965
Poggio, T.: On optimal nonlinear associative recall. Biol. Cybern 19, 201–209 (1975)
Rosenblatt, F.: Principles of neurodynamics; perceptrons and the theory of brain mechanisms. Washington: Spartan Press 1962
Sutton, R.S., Barto, A.G.: Toward a modern theory of adaptive networks: expectation and prediction. Psychol. Rev. (in press) (1981)
Tsetlin, M.L.: Automaton theory and modeling of biological systems. New York: Academic Press 1973
Widrow, B., Gupta, N.K., Maitra, S.: Punish/reward: learning with a critic in adaptive threshold systems. IEEE Trans. Syst. Man Cybern. 5, 455–465 (1973)
Wigström, H.: A neuron model with learning capability and its relation to mechanisms of association. Kybernetik 12, 204–215 (1973)
Willshaw, D.J., Buneman, O.P., Longuet-Higgins, H.S.: Nonholographic associative memory. Nature 222, 960–962 (1969)
Wood, C.C.: Variations on a theme by Lashley: lesion experiments on the neural model of Anderson, Silverstein, Ritz, and Jones. Psychol. Rev. 85, 582–591 (1978)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Barto, A.G., Sutton, R.S. & Brouwer, P.S. Associative search network: A reinforcement learning associative memory. Biol. Cybern. 40, 201–211 (1981). https://doi.org/10.1007/BF00453370
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00453370