Abstract
According to Hebb's postulate for learning, information presented to a neural net during a learning session is stored in the synaptic efficacies. Long-term potentiation occurs only if the postsynaptic neuron becomes active in a time window set up by the presynaptic one. We carefully interpret and mathematically implement the Hebb rule so as to handle both stationary and dynamic objects such as single patterns and cycles. Since the natural dynamics contains a rather broad distribution of delays, the key idea is to incorporate these delays in the learning session. As theory and numerical simulation show, the resulting procedure is surprisingly robust and faithful. It also turns out that pure Hebbian learning is by selection: the network produces synaptic representations that are selected according to their resonance with the input percepts.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Amit DJ (1987) Neural networks counting chimes. RIP Preprint RI/87/49
Amit DJ, Gutfreund H, Sompolinsky H (1985) Spin-glass models of neural networks. Phys Rev A 32:1007–1018
Amit DJ, Gutfreund H, Sompolinsky H (1987) Statistical mechanics of neural networks near saturation. Ann Phys (NY) 173:30–67
Bös S (1988) Neuronales Netzwerk mit hierarchisch strukturierter Information. Diplomarbeit, Universität Heidelberg
Braitenberg V (1967) Is the cerebellar cortex a biological clock in the millisecond range? In: Fox CA, Snider RS (eds) The cerebellum. Progress in Brain Research, vol 25. Elsevier, Amsterdam, pp 334–346
Braitenberg V (1974) On the representation of objects and their relation in the brain. In: Conrad M, Güttinger W, Dal Cin M (eds) Lecture Notes in Biomathematics, vol 4. Springer, Berlin Heidelberg New York, pp 290–298
Braitenberg V (1986) Two views of the cerebral cortex. In: Palm G, Aertsen A (eds) Brain theory. Springer, Berlin Heidelberg New York, pp 81–96
Buhmann J (1985) Mustererkennung in selbstorganisierenden, neuronalen Netzwerken. Diplomarbeit. Technische Universität München
Buhmann J, Schulten K (1987) Noise-driven temporal association in neural networks. Europhys Lett 4:1205–1209
Caianiello ER (1961) Outline of a theory of thought processes and thinking machines. J Theor Biol 1:204–235
Coolen ACC, Gielen CCAM (1988) Delays in neural networks. Europhys Lett 7:281–285
Dehaene S, Changeux J-P, Nadal J-P (1987) Neural networks that learn temporal sequences by selection. Proc Natl Acad Sci USA 84:2727–2731
Feigel'man MV, Ioffe LB (1987) The augmented models of associative memory: asymmetric interaction and hierarchy of patterns. Int J Mod Phys 1:51–68
Fukushima K (1973) A model of associative memory in the brain. Kybernetik 12:58–63
Grensing D, Kühn R (1986) Random-site spin glass models. J Phys A 19:L1153–1157
Grossberg S (1968) Prediction theory for some nonlinear functional differential equations I. Learning of lists. J Math Anal Appl 21:643–694
Gustafsson B, Wigström H, Abraham WS, Huang Y-Y (1987) Long-term potentiation in the hippocampus using depolarizing current pulses as the conditioning stimulus to single volley synaptic potentials. J Neurosci 7:774–780
Gutfreund H, Mézard M (1988) Processing of temporal sequences in neural networks. Phys Rev Lett 61:235–238
Hebb DO (1949) The organization of behavior. Wiley, New York
Hemmen JL van (1987) Nonlinear neural networks near saturation. Phys Rev A 36:1959–1962
Hemmen JL van, Kühn R (1986) Nonlinear neural networks. Phys Rev Lett 57:913–916
Hemmen JL van, Grensing D, Huber A, Kühn R (1986) Elementary solution of classical spin-glass models. Z Phys B-Condensed Matter 65:53–63
Hemmen JL van, Grensing D, Huber A, Kühn R (1988a) Nonlinear neural networks I. General theory; II. Information processing. J Stat Phys 50:231–257; 259–293
Hemmen JL van, Keller G, Kühn R (1988b) Forgetful memories. Europhys Lett 5:663–668
Herz A (1989) Representation and recognition of spatiotemporal objects within a generalized Hopfield scheme. Proc Connectionism in Perspective (Zürich): (to be published)
Herz A, Sulzer B, Kühn R, Hemmen JL van (1988) Hebbian learning — a canonical way of representing static and dynamic objects in an associative network. Proc nEuro '88 (Paris): (to be published)
Hopfield JJ (1982) Neural networks and physical systems with emergent collective computational abilities. Proc Natl Acad Sci USA 79:2554–2558
Kelso SR, Ganong AH, Brown TH (1986) Hebbian synapses in hippocampus. Proc Natl Acad Sci USA 83:5326–5330
Kerszberg M, Zippelius A (1988) Synchronization in neural assemblies. Preprint
Kleinfeld D (1986) Sequential state generation by model neural networks. Proc Natl Acad Sci USA 83:9469–9473
Lamperti J (1966) Probability. Benjamin, New York. For non-random patterns one simply requires that p N(y) converge to a limit p(y)
Lee KH, Chung K, Chung JM, Coggeshall RE (1986) Correlation of cell body size, axon size, and signal conduction velocity for individually labelled dorsal root ganglion cells in the cat. J Comp Neurol 243:335–346
Little WA (1974) The existence of persistent states in the brain. Math Biosci 19:101–120
Little WA, Shaw GL (1978) Analytic study of the memory storage capacity of a neural network. Math Biosci 39:281–290
McCulloch WC, Pitts W (1943) A logical calculus of the ideas immanent in nervous activity. Bull Math Biophys 5:115–133
Malinow R, Miller JP (1986) Postsynaptic hyperpolarization during conditioning reversibly blocks induction of long-term potentiation. Nature 320:529–530
Miller R (1987) Representation of brief temporal patterns, Hebbian synapses, and the left-hemisphere dominance for phoneme recognition. Psychobiology 15:241–247
Nadal J-P, Toulouse G, Changeux J-P, Dehaene S (1986) Networks of formal neurons and memory palimpsests. Europhys Lett 1:535–542
Parisi G (1986) A memory which forgets. J Phys A: Math Gen 19:L617–620
Peretto P, Niez JJ (1986) Collective properties of neural networks. In: Bienenstock E, Fogelman-Soulié F, Weisbuch G (eds) Disordered systems and biological organization. Springer, Berlin Heidelberg New York, pp 171–185
Riedel U, Kühn R, Hemmen JL van (1988) Temporal sequences and chaos in neural nets. Phys Rev A 38:1105–1108
Scott AC (1977) Neurophysics. Wiley, New York
Sompolinsky H, Kanter L (1986) Temporal association in asymmetric neural networks. Phys Rev Lett 57:2861–2864
Toulouse G, Dehaene S, Changeux J-P (1986) A spin-glass model of learning by selection. Proc Natl Acad Sci USA 83:1695–1698
Willwacher G (1976) Fähigkeiten eines assoziativen Speicher-systems im Vergleich zu Gehirnfunktionen. Biol Cybern 24:181–198
Willwacher G (1982) Storage of a temporal pattern sequence in a network. Biol Cybern 43:115–126
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
Herz, A., Sulzer, B., Kühn, R. et al. Hebbian learning reconsidered: Representation of static and dynamic objects in associative neural nets. Biol. Cybern. 60, 457–467 (1989). https://doi.org/10.1007/BF00204701
Received:
Issue Date:
DOI: https://doi.org/10.1007/BF00204701