Elsevier

Neural Networks

Volume 62, February 2015, Pages 3-10
Neural Networks

2015 Special Issue
Mathematical modeling for evolution of heterogeneous modules in the brain

https://doi.org/10.1016/j.neunet.2014.07.013Get rights and content

Abstract

Modular architecture has been found in most cortical areas of mammalian brains, but little is known about its evolutionary origin. It has been proposed by several researchers that maximizing information transmission among subsystems can be used as a principle for understanding the development of complex brain networks. In this paper, we study how heterogeneous modules develop in coupled-map networks via a genetic algorithm, where selection is based on maximizing bidirectional information transmission. Two functionally differentiated modules evolved from two homogeneous systems with random couplings, which are associated with symmetry breaking of intrasystem and intersystem couplings. By exploring the parameter space of the network around the optimal parameter values, it was found that the optimum network exists near transition points, at which the incoherent state loses its stability and an extremely slow oscillatory motion emerges.

Introduction

Modular architecture is one of the most frequently referenced concepts of neural organization (Felleman and Van Essen, 1991, Mountcastle, 1997, Szentágothai, 1983). In biological systems, such modular architecture has been self-organized in ontogeny, influenced by phylogeny (Wagner, Pavlicev, & Cheverud, 2007). In contrast, the modules and their interactions of artificial machines are designed by humans. Understanding the mechanism giving rise to functionally differentiated modules in biological systems, such as cortical module architectures, is of great interest.

In theoretical studies, it has been proposed that maximizing information transmission between subsystems can be used as a guiding principle for understanding the development and evolution of complex brain networks. Linsker, 1988, Linsker, 1989, Linsker, 1997 shows that information transmission between successive layers of feed-forward networks is a viable principle for designing functional neural networks. Recently, Tanaka, Kaneko, and Aoyagi (2009) showed that the learning algorithm that maximizes information retention in recurrent networks also gives rise to the appearance of biological structures, such as cell assemblies, and even to dynamics, such as spontaneous activity of synfire chain and critical neuronal avalanches. One of the present authors has proposed the idea of a new self-organization principle based on a variational principle (Kaneko and Tsuda, 2001, Tsuda, 1984, Tsuda, 2001): components (or elements) are self-organized according to constraints that act on the whole system, whereas according to the usual self-organization principles (ex.  Haken, 1980, Nicolis and Prigogine, 1977), macroscopic order is self-organized via interactions among microscopic elements. Actually, neuron-like elements as components of a system have been obtained under the condition of maximum transmission of information across a whole system (Ito and Tsuda, 2007, Watanabe et al., in preparation). In the brain, bidirectional information transmission among modules is crucial for information processing. In fact, most connections between cortical modules are known to be bidirectional (Felleman & Van Essen, 1991). Yamaguti, Tsuda, and Takahashi (2014) have found intermittent switching of direction of information flow in two heterogeneously coupled chaotic systems. Such a mechanism may be applicable to bidirectional information transmission in neural systems.

Motivated by these studies, here we propose a mathematical model for functional differentiation induced by selection based on maximizing bidirectional information transmission. We try to extract the essence of evolutionary dynamics by investigating a coupled-oscillator network model. We construct randomly coupled-oscillator networks, each consisting of two sub-networks, by using a discretized version of the Kuramoto phase oscillator model (Kuramoto, 1984). Each sub-network consists of N oscillators, whose states are represented by phase variables. A genetic algorithm is used to modify the parameters to the direction of better fitness. Transfer entropy (Schreiber, 2000), which measures directed information transfer, is estimated to quantify information flow between the two sub-networks in both directions and their product is regarded as fitness in the evolutionary process.

In Section  2, the proposed network model, analytic method, and procedure for evolution by the genetic algorithm are described. In Section  3, we give numerical results of this evolutionary process. In Section  4, we study the fitness landscape around the optimum network, exploring the parameter space as a means of characterizing the evolutionary process from the viewpoint of dynamical systems. In Section  5, dependences of the coupling density on heterogeneity are investigated. Section  6 is devoted to summary and discussions.

Section snippets

Network model

We consider a network of oscillators, which consists of two sub-network systems. Each sub-network consists of N=200 phase oscillators, whose dynamics is described by a discrete-time version of the Kuramoto model (Barlev et al., 2010, Daido, 1986, Kuramoto, 1984). It is known that the dynamics of weakly coupled oscillators that have stable limit-cycles can be reduced to a coupled equation of phase oscillators by using a reduction technique (Kuramoto, 1984). The Kuramoto model corresponds to the

Evolution of the network

We numerically simulated network evolution. In this section, the result for p=0.05 is shown. By the end of simulation (1000 generations), the mean fitness of the selected networks was almost saturated (Fig. 2(A)). In all simulations, genome populations were distributed around their mean values in a single cluster; split populations were not observed at the end of the evolution. The parameter values of the network at the end of the evolution and its schematic diagram are summarized in Fig. 2.

Fitness landscape around the optimized network

In this section, we explored the fitness landscape around the network with the optimal parameters found in the previous section. We investigate landscapes of two-dimensional spaces spanned by two control parameters, fixing other parameters in the vicinity of optimal values.

Fig. 5(A) shows the fitness landscape of w(11)w(22) parameter space, where w(ij)=p1(ij)p2(ij). Other parameter values are fixed as follows: q=0.6, r=0.7, w(12)=1.0, and w(21)=1.0. Fig. 5(B) shows means and standard

Dependence on structural inhomogeneity

How does the emergence of heterogeneous modules depend on the coupling density? In this section, we investigate the dependence of the emergence of heterogeneous modules on sparseness of couplings.

Discussion

Development of two interacting sub-networks by an evolutionary algorithm under the constraint of maximizing bidirectional information transmission was studied. Emergence of differentiated modules was demonstrated. By exploring the parameter space of the network around the optimum set of parameter values, it was found that optimal networks exist near transition points, at which the incoherent state loses its stability and an extremely slow oscillatory behavior emerges. Our results support the

Acknowledgments

This work was partially supported by a Grant-in-Aid for Scientific Research on Innovative Areas “The study on the neural dynamics for understanding communication in terms of complex hetero systems (No. 4103)” (21120002) of the Ministry of Education, Culture, Sports, Science and Technology, Japan, by the Human Frontier Science Program (RGP0039/2010), and by JSPS KAKENHI Grant (No. 26540123). The authors would like to thank the anonymous reviewers for giving valuable comments that improved this

References (40)

  • K. Friston et al.

    Analysing connectivity with Granger causality and dynamic causal modelling

    Current Opinion in Neurobiology

    (2013)
  • A. Kaiser et al.

    Information transfer in continuous processes

    Physica D

    (2002)
  • C. Molter et al.

    Rhythmic modulation of theta oscillations supports encoding of spatial and behavioral information in the rat hippocampus

    Neuron

    (2012)
  • G. Barlev et al.

    Map model for synchronization of systems of many coupled oscillators

    Chaos

    (2010)
  • L. Barnett

    Granger causality and transfer entropy are equivalent for Gaussian variables

    Physical Review Letters

    (2009)
  • A.J. Bell et al.

    An information-maximization approach to blind separation and blind deconvolution

    Neural Computing

    (1995)
  • A. Brovelli et al.

    Beta oscillations in a large-scale sensorimotor cortical network: directional influences revealed by Granger causality

    Proceedings of the National Academy of Sciences of the United States of America

    (2004)
  • J. Clune et al.

    The evolutionary origins of modularity

    Proceedings of the Royal Society B: Biological Sciences

    (2013)
  • H. Daido

    Discrete-time population dynamics of interacting self-oscillators

    Progress of Theoretical Physics

    (1986)
  • C. Espinosa-Soto et al.

    Specialization can drive the evolution of modularity

    PLoS Computational Biology

    (2010)
  • D. Felleman et al.

    Distributed hierarchical processing in the primate cerebral cortex

    Cerebral Cortex

    (1991)
  • M.D. Fox et al.

    The human brain is intrinsically organized into dynamic, anticorrelated functional networks

    Proceedings of the National Academy of Sciences of the United States of America

    (2005)
  • C. Granger

    Investigating causal relations by econometric models and cross-spectral methods

    Econometrica: Journal of the Econometric Society

    (1969)
  • H. Haken

    Synergetics

    Naturwissenschaften

    (1980)
  • J.H. Holland

    Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, and artificial intelligence

    (1975)
  • T. Ito et al.

    Propagation and storage of input information in the network with feedback

    Bussei Kenkyu

    (2007)
  • K. Kaneko et al.

    Complex systems: chaos and beyond: a constructive approach with applications in life sciences

    (2001)
  • N. Kashtan et al.

    Spontaneous evolution of modularity and network motifs

    Proceedings of the National Academy of Sciences of the United States of America

    (2005)
  • Y. Kuramoto

    Chemical oscillations, waves, and turbulence

    (1984)
  • P. Lakatos et al.

    Entrainment of neuronal oscillations as a mechanism of attentional selection

    Science

    (2008)
  • Cited by (25)

    • Spike signal transmission between modules and the predictability of spike activity in modular neuronal networks

      2021, Journal of Theoretical Biology
      Citation Excerpt :

      Many researches indicate that modules are crucial for promoting and preserving functional specialization in neuronal networks (Espinosa-Soto and Wagner, 2010; Gallos et al., 2012). In many computational simulations, modules exhibit the capability of enriching distribution patterns of spike signal in neuronal networks (Yamaguti and Tsuda, 2015). In addition, modules also play important roles in the dynamics of neuronal networks, such as promoting meta-stability and synchronization (Arenas et al., 2006; Wildie and Shanahan, 2012).

    • Spectral peculiarity and criticality of a human connectome

      2019, Physics of Life Reviews
      Citation Excerpt :

      Since any global brain function is a synergetic collective effect of many particular functions, it should be treated by appropriate methods capable to catch collective properties of underlying structural network. From this point of view, the comparative investigation of different connectomes undertaken in [20] via the analysis of their spectral properties, seems particularly promising (see also [22]). In our work we continue this line of reasoning and develop the comparative spectral approach.

    • Altered brain connectivity in sudden unexpected death in epilepsy (SUDEP) revealed using resting-state fMRI

      2019, NeuroImage: Clinical
      Citation Excerpt :

      Patterns of centrality alterations, an increase in the number of hubs (hub prevalence), and reorganization of hubs among regulatory structures differentiated low-risk patients from high-risk and SUDEP cases, suggesting different pathways to functional reorganization that are related to SUDEP risk. A clear division of a network into modules (assessed with modularity) is a prominent feature of many biological networks, including the mammalian cortical architecture (Yamaguti and Tsuda, 2015). Decreased modularity of resting-state functional brain networks occurs in several pathological states, e.g., childhood-onset schizophrenia (Alexander-Bloch et al., 2010), and with poorer responses to cognitive training in aging adults following traumatic brain injury (Gallen et al., 2016).

    View all citing articles on Scopus
    View full text