Abstract
An approach to solving nonlinear control problems is illustrated by means of a layered associative network composed of adaptive elements capable of reinforcement learning. The first layer adaptively develops a representation in terms of which the second layer can solve the problem linearly. The adaptive elements comprising the network employ a novel type of learning rule whose properties, we argue, are essential to the adaptive behavior of the layered network. The behavior of the network is illustrated by means of a spatial learning problem that requires the formation of nonlinear associations. We argue that this approach to nonlinearity can be extended to a large class of nonlinear control problems.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Albus, J.S.: Mechanisms of planning and problem solving in the brain. Math. Biosci.45, 247–293 (1979)
Amari, S., Arbib, M.A.: Competition and cooperation in neural nets. In: Systems neuroscience. Metzler, J. (ed.). New York: Academic Press 1977
Arbib, M.A.: Segmentation, schemas and cooperative computation. In: Studies in mathematical biology. Part I. Cellular behavior and the development of pattern. Levin, S. (ed.). Math. Ass. Am. 1978
Arbib, M.A.: Perceptual structures and distributed motor control. In: Handbook of physiology. Motor control. Vol. III. Brooks, V.B. (ed.). Bethesda, MD: Am. Physiol. Soc. 1981 (to appear)
Barto, A.G., Sutton, R.S.: Landmark learning: an illustration of associative search. Biol. Cybern.42, 1–8 (1981a)
Barto, A.G., Sutton, R.S.: Goal-seeking components for adaptive intelligence: an initial assessment. Technical Report AFWALTR-81-1070, Avionics Laboratory, Air Force Wright Aeronautical Laboratories, Wright-Patterson Air Force Base, Ohio (1981b)
Barto, A.G., Sutton, R.S., Brouwer, P.: Associative search network: a reinforcement learning associative memory. Biol. Cybern.40, 201–211 (1981)
Didday, R.L.: A model of visuomotor mechanisms in the frog optic tectum. Math. Biosci.30, 169–180 (1976)
Duda, R.O., Hart, P.E.: Pattern classification and scene analysis. New York: Wiley 1973
Fukushima, K.: A model of associative memory in the brain. Kybernetik12, 58–63 (1973)
Fukushima, K.: Neccognitron: A self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol. Cybern.36, 193–202 (1980)
Grossberg, S.: Adaptive pattern classification and universal recoding. I. Parallel development and coding of neural feature detectors. Biol. Cybern.23, 121–134 (1976a)
Grossberg, S.: Adaptive pattern classification and universal recoding. II. Feedback,-expectation, olfaction, illusions. Biol. Cybern.23, 187–202 (1976b)
Ivakhnenko, A.G.: Polynomial theory of complex systems. IEEE Transactions on Systems, Man, and Cybernetics SMC-1, 364–378 (1971)
Kilmer, W.L., McCulloch, W.S., Blum, J.: A model of the vertebrate central command system. Int. J. Man-Machine Stud.1, 279–309 (1969)
Klopf, A.H., Gose, E.: An evolutionary pattern recognition network. IEEE Trans. Syst. Sci. Cybern.5, 247–250 (1969)
Klopf, A.H.: Brain function and adaptive systems — a heterostatic theory. Air Force Cambridge Research Laboratories Research Report AFCRL-72-0164, Bedford, MA, 1972. (A summary appears in: Proc. Int. Conf. Syst., Man, Cybern, IEEE Syst. Man, Cybern. Soc. Dallas, Texas, 1974)
Klopf, A.H.: Goal-seeking systems from goal-seeking components: implications for AI. Cogn. Brain Theor. Newslett.3, 2 (1979)
Klopf A.H.: The hedonistic neuron: a theory of memory, learning, and intelligence. Washington, D.C.: Hemisphere Publishing Corp. 1982 (to be published)
Mendel, J.M., McLaren, R.W.: Reinforcement-learning control and pattern recognition systems. In: Adaptive, learning, and pattern recognition systems: theory and applications, pp. 287–317. Mendel, J.M., Fu, K.S. (eds.). New York: Academic Press 1970
Michie, D., Chambers, R.A.: BOXES: an experiment in adaptive control. Machine intelligence 2, pp. 137–152. Dale, E., Michie, D. (eds.). Edinburgh: Oliver and Boyd 1968
Minsky, M.L.: Steps toward artificial intelligence. Proc. IRE49, 8–30 (1961)
Minsky, M.L.: A framework for representing knowledge. In: The psychology of computer vision, pp. 211–277. Winston, P.H. (ed.). New York: McGraw-Hill 1975
Minsky, M.L., Papert, S.: Perceptrons: an introduction to computational geometry. Cambridge, MA: MIT Press 1969
Nilsson, N.J.: Learning machines. New York: McGraw-Hill 1965
Poggio, T.: On optimal nonlinear associative recall. Biol. Cybern.19, 201–209 (1975)
Raibert, M.H.: A model for sensorimotor control and learning. Biol. Cybern.29, 29–36 (1978)
Rosenblatt, F.: Principles of neurodynamics. New York: Spartan Books 1962
Selfridge, O.G.: Pattern recognition and modern computers. In: Proceedings of the 1955 Western Joint Computer Conference, Session on Learning Machines, W.H. Ware Chairman, 91–93 (1955)
Spinelli, D.N.: OCCAM: A computer model for a content addressable memory in the central nervous system. In: The biology of memory. Pribram, K., Broadbent, D. (eds.). New York: Academic Press 1970
Sutton, R.S., Barto, A.G.: Toward a modern theory of adaptive networks: Expectation and prediction. Psychol. Rev.88, 135–170 (1981)
von der Malsburg, C.: Self-organization of orientation sensitive cells in the striate cortex. Kybernetik14, 85–100 (1973)
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Barto, A.G., Anderson, C.W. & Sutton, R.S. Synthesis of nonlinear control surfaces by a layered associative search network. Biol. Cybernetics 43, 175–185 (1982). https://doi.org/10.1007/BF00319977
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00319977