The Model of the Cybernetic Network and Its Realization on the Cluster of Universal and Graphic Processors

In recent years, the huge attention is paid to questions of controllability by networks in a classical statement at which the operating signals influence a row, so-called, leading knots, and variables of a vector of a network’s condition are connected with its hubs [1‐3]. So, for example in [4] influence of attacks on the control of the network is investigated, and it is revealed that the purposeful attack on links of knots is more effective than the stochastic attack to knots. In addition to it in works [5, 6], the new approach in which also dynamics of links is considered which is connected with vectors of a condition of the network.

It seems appropriate to consider the network model with built-in controllers for controlling various connections, as well as its implementation on a cluster-purpose and graphics processors.

The proposed model and its software implementation may be used to solve quite a wide range of tasks of network dynamics, segmentation and recognition signal and image processing, signal processing phased arrays, the simulation of hardware implementations of networks, technologies process control.

In this paper, we solve the problem of modeling on the functional–structural level the multiple heterogeneous network consisting of two subnets (information and management), formed by a large number of active elements.

The ultimate goal is to realize about a complex network model on a cluster of universal and GPUs to create high-performance, scalable tool for solving various problems of network dynamics, segmentation, data compression, and pattern recognition.

The basic idea of the structural organization of the network, in terms of the introduction of a management subnet, is borrowed from neurobiological studies [6‐9].

However, compared with the existing approaches, which describes only the idea of the flow of calcium metabolism and neurotransmitter between synapses and astrocytes, we propose a cybernetic generalization in terms of the interaction between two subnets, one of which is formed by connecting switches and their connectors, and the other is formed by controllers. In the model, each controller polls a few connectors, to give information about their conditions, then adjusts their parameters [10, 11].

The introduction of the controller’s subnet allows experimentally examined the relationship issues of governance networks, the formation of a variety network clusters, their influence on the processes of learning networks, as well as, to increase their operational reliability. Using of multiple controllers can also parallelize the control process for networks with a large number of connections (≥10⁸).

The proposed network includes following subnets and their structural elements.

Information subnet formed by commutators («Comm», N) and connectors («Conn», N × N).

Commutator (switch) «Comm» has input and output communication ports connected hubs with internal and external inputs and outputs, as well as ports of external control.

Connector «Conn» communicates between switches and has input and output communication ports, as well as ports of external and internal control.

Management subnet formed by controllers («Contr», M ≥ N).

Controllers «Contr» have input and output communication ports as well as ports of external and internal control. These ports are connected via communication hubs channels.

Setting up the structural elements of a heterogeneous network carried out via the external control trunk—Trunk of an external control (Fig. 1).

The informational subnet is an exchange of information data flows I _comm and I _conn, convertible switches and connectors thru the corresponding Trunks of switches and connectors.

The input of each switch «Comm» via internal Trunk of switches is connected to the output of the group of connectors «Conn».

The information output of each switch «Comm» is connected via internal Trunks of switches with a group of connectors. For example, the output of the switch Comm₁—the information data flow Icomm₁ effects on inputs of connectors Conn_1,1, …, Conn_{1, N}.

In the management subnet, the exchange by streams of S _conn and C data via the Trunk1, Trunk2 of internal management is occurred (Fig. 1). Data flow S _conn describes the state of connectors and C describes the control by connectors from the controllers.

The exchange by information data flows I _contr between all controllers carried out via the internal Trunk₃ (Fig. 1).

Since a number of M controllers are much smaller than the number of N x N connectors, each controller controls a group (set) of connectors. For example, connectors of the group {1,1}, …, {1, N} (Conn_1,1, …, Conn_{1, N} ) are operated by controller Contr₁ which generally performs various controls C _1,1, …, C _1,N . In turn, each m-th controller receives information S _{m}, which characterizes the state of a group {m} of connectors connected to it (m = 1, …, M).

Structural elements of the network discussed above have the following active functions.

To describe the transfer functions of its element as static Boolean expressions, as dynamic equations discussed, for example, [3, 12] can be used. In the latter case, in the n-th switch Comm _n the integration of the dynamic equations system is carried out. In this case, the specific form of the dynamic equation and its parameters are set via an external control line (Fig. 1).

For the purposes of neurodynamic the condition of formation the value I _comm signal output can be set, for example, in the form of: if I _comm ≥ I _thr, I _out; otherwise I _comm = 0, where I _thr—the signal threshold.

To describe transfer functions of the element as static Boolean expressions, dynamic equation discussed, for example, in [14] can be used. In the latter case, in k, l-th connector Conn _k,l (k, l = 1, 2, …, N × N), connecting k-th and l-th switches (see Fig. 1), integration of the dynamic equations was carried out describing the rate of change I _conn(k,l) output information signal and the speed change of the function S _conn(k,l) its state depending on the input information signal I _comm(k) from the kth switch and control signal C _cont r(m) from the m-th controller (m = 1, …, M; N ≤ M ≪ N × N).

To describe its operation as static Boolean expressions, dynamic equation is discussed, for example, in [12, 13] can be used. In the latter case, in the m-th controller Contr _m , the integration of dynamic equations is performed.

Input control signal for the controller is the function S _conn (k, l) of the state of the polled k, l-th connector, and the output control signal is the function C _contr(k,l). Communication takes place via a line Trunk of the internal control (Fig. 1).

Each m-th controller produces an output information signal I _contr(m) depending on information signals I _contr(m-k), I _contr(m + k), associated through line Trunk of the communication exchange. The contact for information channels needed for the backup of control functions in case of the failure of any controllers. Using multiple controllers we can also parallelize the process of the control for networks with a large number of links.

For a network with a large number of active elements (N ≥ 10⁵) there are modern technologies based on graphics processors (GPU) and cluster universal processors (CPU).

The following is considered as an object-oriented approach to software implementation of our model on a cluster of general purpose processors and graphics cards.

The library for the modeling of elements of the multiply heterogeneous network of active elements comprises the following groups of classes:

Each class uses software and hardware solutions for the implementation of parallel processing.

The class of Hierarchies is built using object-oriented programming principles. Communicator Groups, Connector, and Controller encapsulate the basic interfaces for their classes.

To create a class containing specific implementation of methods associated with the implementation of specific mathematical models for the specific architecture, the inheritance diagram (Fig. 2) is used.

In this figure, the creation of specific heirs of the Comms class to carry on the CPU and GPU cluster is presented.

The Comms class provides a common interface for all classes, modeling the job of the switch. He is the ancestor for CommCPU and CommGPU classes that contain specialized structures for the data storing for universal and graphics processors.

To develop specific classes that implement specific mathematical models, a base class SomeComms, containing fields and methods necessary to implement the necessary mathematical model is created. Then the classes SomeCommsCPU and SomeCommsGPU, performing calculations on universal and graphics processors are developed.

The polymorphism principle, adopted in object-oriented programming, enables the development of algorithms, working with pointers on base classes, and the network is programmed using pointers on base classes, which in the course of the implementing of various scenarios of the program can be transferred any heirs (Fig. 3).

Figure reflects the cyclical exchange of information (I _comm, I _conn) and managers (S _conn, C _contr) data streams.

Each of the classes CommsPerNode, ConnsPerNode, and ContrsPerNode engaged in processing of portion of objects fall on one cluster node. The latter enables the simulation of the work of large networks on a cluster of universal and (or) GPUs.

As an example, consider following the solution of problems of the modeling the neurodynamic process with the using of the cluster based on Intel Core i5-2300 processor and graphics GM206 CPUs (NVidia Geforce GTX 950).

Experiments with the network of switches. To test the developed object-oriented approach as a switch «Comm» the neuron dynamic equation in the form of E.M. Izhikevich model [14] was chosen, as a connector «Conn» the synapse with freewheeling and adjustable the amplitude response and the time τ_conn of its fall was chosen, as a controller «Contr» the astrocyte with the control function of the amplitude of the time τ_conn of the connector signal recession was chosen.

The process of generation of excitation (spikes) of 100 neurons under their relationship in the form of a complete graph and different settings of relaxation times τ_conn of synaptic responses was investigated.

Figure 4 shows the activity of neurons, divided into groups (34, 33, 33) with the relaxation times of synaptic responses 0.4, 4, and 40 ms, respectively.

The diagram shows that neurons with long relaxation times of synaptic responses often generate spikes.

At the additional testing, the fully connected network of 1000 neurons was used. The model of E.M. Izhikevich built on the basis of a mathematical model implemented in Matlab package and realized by numerical Euler method was used as the referent set [15]. The results of this model were compared with the results obtained by us on the GPU and CPU (Fig. 5).

In simulation experiments as well as in [15], the alpha rhythm (10 Hz) and gamma wave (40 Hz) were observed. Thereby, the correct operation of the developed model and its implementation, both graphical and universal processors were confirmed.

Experiments on the interaction of subnets switches and controllers. During testing of the developed model, the interaction between the network of connectors Conns and the network of controllers Contr was qualitatively investigated. In the first experiment as a model of switches Comms, the model of a neuron «Integrate and Fire» [15] resistant to interference was taken, as well as models of interaction between the synapse and astrocyte through the sharing of neurotransmitter of works [7‐9] were taken.

In the first experiment, switches (neurons) connected to each other through connectors, which are connected to the controller (Fig. 6).

On the external information inputs of switches excitation pulses were supplied and to their outputs has been added the additive evenly distributed interference that passed through connectors to internal information inputs of switches.

In the absence of the control loop and under exceeding by exciting pulses, the threshold value switches generated stable spikes on the entire time interval of exciting pulses (0 ÷ 100 ms) regardless of changes of the interference amplitude over a wide range (0 ÷ 5 μA/cm²).

After the control loop switch on for one of the links functioning according to regulatory models of neuro- and gliatransmitter exchange [7‐9], for some implementations of interference breakdowns generate switch connected to this connection are occurring.

Figures 7 and 8 are examples the effect of different implementations of noise (0.5 μA/cm²) at various power of the communication between connectors and controllers.

In a second experiment, the effect of controllers on the connectors in the network with the correct regular topology (complete graph), which is most resistant to variations of relations [4] was studied. As a dynamic model of the switches functioning the model of «Izhikevich» neuron [14, 15] was used. For example, in diagram (Fig. 9a) all weight is the same and equal to 100, and in the diagram (Fig. 9b) for 10% of bonds weights are set equal to 1.

Carried out simulations showed that the developed model of the cyber network and object-oriented approach to its software implementation helped to create a very flexible and versatile tool for the solving problems of neurodynamics.

For example, our results of the network simulation coincided with known results of E.M Izhikevich. Also it shows qualitatively the fundamental possibility of the control for the network of commutators by the network of controllers.

The use of GPUs and the CUDA parallel programming technology from NVIDIA shows better performance on large data volumes, compared with the universal processors because of their greater efficiency for the use of computing concepts SIMD (Single Instruction Multiple Data), even without the use of optimization mechanisms.

The investigation of the effect of these mechanisms on the performance of the model under consideration and test it on a modern line of GPU (NVIDIA Pascal) are planned in the course of this work.

In further, the considered model and its advanced software implementation will be used for the experimental study of the stability of the dynamics of complex networks near the critical parameters of their relationship values, handling complex networks.