The gamma model—A new neural model for temporal processing

doi:10.1016/S0893-6080(05)80035-8

Neural Networks

Volume 5, Issue 4, July–August 1992, Pages 565-576

https://doi.org/10.1016/S0893-6080(05)80035-8 Get rights and content

Abstract

In this paper we develop the gamma neural model, a new neural net architecture for processing of temporal patterns. Time varying patterns are normally segmented into a sequence of static patterns that are successively presented to a neural net. In the approach presented here segmentation is avoided. Only current signal values are presented to the neural net, which adapts its own internal memory to store the past. Thus, in the gamma neural net, an adaptive short term mechanism obviates a priori signal segmentation. We evaluate the relation between the gamma net and competing dynamic neural models. Interestingly, the gamma model brings many popular dynamic net architectures, such as the time-delay-neural-net and the concentration-in-time-neural-net, into a unifying framework. In fact, the gamma memory structure appears as general as a temporal convolution memory structure with arbitrary time varying weight kernel w(t). Yet, the gamma model remains mathematically equivalent to the additive (Grossberg) model with constant weights. We present a back propagation procedure to adapt the weights in a particular feedforward structure, the focused gamma net.

References (27)

BodenhausenU. et al.
Learning the architecture of neural networks for speech recognition
BraunM.
Ordinary differential equations and their applications
(1983)
de VriesB. et al.
A theory for neural nets with time delays
ElmanJ.L.
Finding structure in time
Cognitive Science
(1990)
FarqueM.D.
Reductibilite des systemes hereditaires a des systemes dynamiques (regis par des equations differentielles ou aux derivees partielles)
C.R. Acad. Sc. Paris
(1973)
FukushimaK.
Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition by shift in position
Biological Cybernetics
(1980)
GoriM. et al.
BPS: A learning algorithm for capturing the dynamic nature of speech
GrossbergS.
Studies of mind and brain
(1982)
JordanM.I.
Atractor dynamics and parallelism in a connectionist sequential machine
HertzJ. et al.
Introduction to the theory of neural computation
(1991)

LangK. et al.

A time-delay neural network architecture for isolated word recognition

Neural Networks

(1990)

MillerR.

Representation of brief temporal patterns, Hebbian synapses, and the left-hemisphere dominance for phoneme recognition

Psychobiology

(1987)

MozerM.C.

A focused back propagation algorithm for temporal pattern recognition

Complex Systems

(1989)

Cited by (196)

Persistence in complex systems
2022, Physics Reports
Persistence is an important characteristic of many complex systems in nature, related to how long the system remains at a certain state before changing to a different one. The study of complex systems’ persistence involves different definitions and uses different techniques, depending on whether short-term or long-term persistence is considered. In this paper we discuss the most important definitions, concepts, methods, literature and latest results on persistence in complex systems. Firstly, the most used definitions of persistence in short-term and long-term cases are presented. The most relevant methods to characterize persistence are then discussed in both cases. A complete literature review is also carried out. We also present and discuss some relevant results on persistence, and give empirical evidence of performance in different detailed case studies, for both short-term and long-term persistence. A perspective on the future of persistence concludes the work.
Hopf bifurcation analysis of a complex-valued neural network model with discrete and distributed delays
2018, Applied Mathematics and Computation
In this paper, a class of complex-valued neural network model with discrete and distributed delays is proposed. Regarding the discrete time delay as the bifurcating parameter, the problem of Hopf bifurcation in the newly-proposed complex-valued neural network model is investigated under the assumption that the activation function can be separated into its real and imaginary parts. Based on the normal form theory and center manifold theorem, some sufficient conditions which determine the direction of the Hopf bifurcation and the stability of the bifurcating periodic solutions are established. Finally, a numerical example is given to illustrate the validity of the theoretical results.
Multistability of delayed neural networks with hard-limiter saturation nonlinearities
2018, Neurocomputing
The paper considers a class of nonsmooth neural networks where hard-limiter saturation nonlinearities are used to constrain solutions of a linear system with concentrated and distributed delays to evolve within a closed hypercube of $R^{n}$ . Such networks are termed delayed linear systems in saturated mode (D-LSSMs) and they are a generalization to the delayed case of a relevant class of neural networks previously introduced in the literature. The paper gives a rigorous foundation to the D-LSSM model and then it provides a fundamental result on convergence of solutions toward equilibrium points in the case where there are nonsymmetric cooperative (nonnegative) interconnections between neurons. The result ensures convergence for any finite value of the maximum delay and is physically robust with respect to perturbations of the interconnections. More importantly, it encompasses situations where there exist multiple stable equilibria, thus guaranteeing multistability of cooperative D-LSSMs. From an application viewpoint the delays in combination with the property of multistability make D-LSSMs potentially useful in the fields of associative memories, motion detection and processing of temporal patterns.
On the Feasibility of Simple Transformer for Dynamic Graph Modeling
2024, arXiv
Persistent learning signals and working memory without continuous attractors
2023, arXiv
Universal Recurrent Event Memories for Streaming Data
2023, arXiv

View all citing articles on Scopus

^*: Current address: David Sarnoff Research Center, CN5300, Princeton, NJ 08543-5300.

View full text

Original ContributionThe gamma model—A new neural model for temporal processing

Abstract

Learning the architecture of neural networks for speech recognition

Ordinary differential equations and their applications

A theory for neural nets with time delays

Finding structure in time

Cognitive Science

Reductibilite des systemes hereditaires a des systemes dynamiques (regis par des equations differentielles ou aux derivees partielles)

C.R. Acad. Sc. Paris

Neocognitron: A self-organizing neural network model for a mechanism of pattern recognition by shift in position

Biological Cybernetics

BPS: A learning algorithm for capturing the dynamic nature of speech

Studies of mind and brain

Atractor dynamics and parallelism in a connectionist sequential machine

Introduction to the theory of neural computation

A time-delay neural network architecture for isolated word recognition

Neural Networks

Representation of brief temporal patterns, Hebbian synapses, and the left-hemisphere dominance for phoneme recognition

Psychobiology

A focused back propagation algorithm for temporal pattern recognition

Complex Systems

Original Contribution
The gamma model—A new neural model for temporal processing