Weakly Connected Neural Networks

In this chapter we give definitions and explanations of basic neurophysio­logical terminology that we use in the book. We do not intend to provide a comprehensive background on various topics.

Bifurcations play important roles in analysis of dynamical systems; they express qualitative changes in behavior. A typical example of a bifurcation in neuron dynamics is the transition from rest to periodic spiking activity. Generation of a single spike is a bifurcation phenomenon too: It can result when the rest potential is close to a bifurcation point, which is often referred to as its threshold value. We discuss these and similar issues in this chapter.

Consider a neuron with its membrane potential near a threshold value. When an external input drives the potential to the threshold, the neuron’s activity experiences a bifurcation: The equilibrium corresponding to the rest potential loses stability or disappears, and the neuron fires. This bifurcation is local, but it results in a nonlocal event — the generation of an action potential, or spike. These are observable global phenomena. To model them requires knowledge about global features of neuron dynamics, which usually is not available. However, to predict the onset of such phenomena, one needs only local information about the behavior of the neuron near the rest potential. Thus, one can obtain some global information about behavior of a system by performing local analysis. Our nonhyperbolic neural network approach uses this observation.

Theoretical studies of the human brain require mathematical models. Unfortunately, mathematical analysis of brain models is of limited value, since the results can depend on particulars of an underlying model; that is, various models of the same brain structure could produce different results. This could discourage biologists from using mathematics and/or mathematicians. A reasonable way to circumvent this problem is to derive results that are largely independent of the model and that can be observed in a broad class of models. For example, if one modifies a model by adding more parameters and variables, similar results should hold.

The local activity of a weakly connected neural network (WCNN) is described by the dynamical system where X _i ∈ ℝ^m, λ ∈ Λ, and ρ ∈ R, near an equilibrium point. First we use the Hartman-Grobman theorem to show that the network’s local activity is not interesting from the neurocomputational point of view unless the equilibrium corresponds to a bifurcation point. In biological terms such neurons are said to be near a threshold. Then we use the center manifold theorem to prove the fundamental theorem of WCNN theory, which says that neurons should be close to thresholds in order to participate nontrivially in brain dynamics. Using center manifold reduction we can substantially reduce the dimension of the network. After that, we apply suitable changes of variables, rescaling or averaging, to simplify further the WCNN. All these reductions yield simple dynamical systems that are canonical models according to the definition given in Chapter 4. The models depend on the type of bifurcation encountered. In this chapter we derive canonical models for multiple saddle-node, cusp, pitchfork, Andronov-Hopf, and BogdanovTakens bifurcations. Other bifurcations and critical regimes are considered in subsequent chapters, and we analyze canonical models in the third part of the book.

In this chapter we study the local dynamics of singularly perturbed weakly connected neural networks of the form where X _i ∈ ℝ ^k Y _i ∈ ℝ ^m are fast and slow variables, respectively; τ is a slow time; and ′ = d/dτ. The parameters ε and μ are small, representing the strength of synaptic connections and ratio of time scales, respectively. The parameters λ ∈ Λ and ρ ∈ R. have the same meaning as in the previous chapter: They represent a multidimensional bifurcation parameter and external input from sensor organs, respectively. As before, we assume that all functions F _i , G _i , which represent the dynamics of each neuron, and all P _i and Q _i which represent connections between the neurons, are as smooth as necessary for our computations.

In previous chapters we studied dynamics of weakly connected networks governed by a system of ordinary differential equations. It is also feasible to consider weakly connected networks of difference equations, or mappings, of the form where the variables X _i ∈ ℝ ^m , the parameters λ ∈ Λ,ρ ∈ R and the functions F _i and G _i have the same meaning as in previous chapters. The weakly connected mapping (7.1) can be also written in the form where X _k is the kth iteration of the variable X. In this chapter we use form (7.1) unless we explicitly specify otherwise.

In this chapter we study a weakly connected network of neurons each having a saddle-node bifurcation on a limit cycle. Such neurons are said to have Class 1 neural excitability. This bifurcation provides an important example of when local analysis at an equilibrium renders global information about the system behavior.

In this chapter we study weakly connected networks of oscillatory neurons. Our basic assumption is that there is a value of λ ∈ Λ such that every equation in the uncoupled system (ε = 0) has a hyperbolic stable limit cycle attractor γ ⊂ ℝ ^m The activity on the limit cycle can be described in terms of its phase \( \theta \in \mathbb{S}^1 \) of oscillation where Ω_i(λ) is the natural frequency of oscillations. The dynamics of the oscillatory weakly connected system (9.1) can also be described in terms of phase variables:

A WCNN of the form near a multiple Andronov-Hopf bifurcation point was shown (Theorem 5.8) to have a canonical model of the form where ′ = d/d•, • is slow time, and b _i , c _ij , d _i , z _i ∈ ℂ. In this chapter we study general properties of this canonical model. In particular, we are interested in the stability of the origin z ₁ = … = z _n = 0 and in the possibility of in-phase and anti-phase locking.

In Section 5.3.3 we showed that any weakly connected neural network of the form at a multiple cusp bifurcation is governed by the canonical model where ′ = d/dτ, τ is slow time, x _i , r _i , b _i c _ij are real variables, and σ _i = ±1. In this chapter we study some neurocomputational properties of this canonical model. In particular, we use Hirsch’s theorem to prove that the canonical model can work as a globally asymptotically stable neural network (GAS-type NN) for certain choices of the parameters b₁,…, b _n . We use Cohen and Grossberg’s convergence theorem to show that the canonical model can also operate as a multiple attractor neural network (MA-type NN).

In this chapter we analyze the canonical models (6.17) and (6.24) for singu­larly perturbed WCNNs at quasi-static saddle-node and quasi-static cusp bifurcations. In particular, we consider the canonical models in the special case .

In this chapter we study the relationship between synaptic organizations and dynamical properties of networks of neural oscillators. In particular, we are interested in which synaptic organizations can memorize and reproduce phase information. Most of our results are obtained for neural oscillators near multiple Andronov-Hopf bifurcations.