A novel hysteretic chaotic neural network and its applications

doi:10.1016/j.neucom.2007.02.002

Neurocomputing

Volume 70, Issues 13–15, August 2007, Pages 2561-2565

https://doi.org/10.1016/j.neucom.2007.02.002 Get rights and content

Abstract

A model of single neuron with chaotic and hysteretic characteristics is proposed. Neural network coupled by such neurons exhibits complex dynamic behaviors. The network is also studied from the viewpoint of optimization. Chaos and hysteresis phenomena make the network have the characteristic of escaping from a local minimum of the energy function, so it can find a global minimum more easily as compared with others. The experimental results show that it has a higher average success rate of obtaining a global optimization solution.

Introduction

Neural networks have shown to be powerful tools for solving optimization problems, particularly NP-hard problems. The electrophysiological experiments of animals have proven that chaos dynamics and hysteresis phenomena that exist in real neurons and neural networks play important roles in neuron activity [7], [17], [5], [15]. It is believed that the investigation of the dynamic characters of neural networks is helpful to understand the memory rules of the brain. Various chaotic neural networks models are proposed nowadays. Yu [21] proposed a novel approach of encryption based on chaotic neural networks with time-varying delay. Iwai et al. [11] investigated the effects of correlation among stored patterns on the associative dynamics in the chaotic neural network model. Potapov et al. [14] considered the problem of creating a robust chaotic neural network. Chen and Aihara [3] have proposed a transiently chaotic neural network (TCNN). Xu et al. [19] have presented a method of introducing several time-independent parameters into the original TCNN model. On the other hand, many hysteretic neural networks are also proposed, for instance, in [2], [8], [16], [18], [20], and their performs are better than some nonhysteretic neural networks. However, no neurons or networks have the hysteretic and chaotic properties simultaneously. In this letter, we introduce a new neuron model with the two properties simultaneously, and complex dynamic behavior is investigated. It is found that the neural network composed of the neurons has the better performance with respect to computational abilities.

Section snippets

A neuron model

The neuron model can be formulated as $y (t + 1) = ky (t) - α [x (t) - I],$ $x (t) = f [y (t)],$ $f (s) = {\begin{matrix} (1 + e^{- c_{1} (s + a)})^{- 1}, & \dot{s} (t - δ t) > 0, \\ (1 + e^{- c_{2} (s - b)})^{- 1}, & \dot{s} (t - δ t) < 0, \end{matrix}$ $\dot{s} (t - δ t) = \lim_{δ t \to 0} (x (t) - x (t - δ t)) / δ t,$ where x(t) denotes the output of neuron at discrete time t, I is the input bias of neuron, and y(t) is the inner state of the neuron. The activation function f( ) is composed of two offset sigmoid function, a and b are the center parameters, c₁ and c₂ are the shape parameters, and α is the self-feedback gain coefficient. The

Hysteretic chaotic neural network

Consider a network composed of the mentioned neurons to solve the optimization problems. The network model is described as $x_{i} (t) = f (y_{i} (t)),$ $y_{i} (t + 1) = k y_{i} (t) + β (\sum_{j = 1, j \neq i}^{n} w_{ij} x_{j} (t) + I_{i}) - α (x_{i} (t) - I_{0}),$ where β is the coupling coefficient among the neurons. With β increasing, the coupling degree among neurons becomes stronger in chaotic status, and the neurons will escape from chaotic states and get into stable states. Tuning the parameter quickens the optimization rate.

Chaos searching has the ergodicity

Numerical experiments

The networks can be applied to the function optimization problem and the combinatorial optimization problem.

Conclusion

In this letter, we propose a novel neuron model whose activation function is composed of two offset sigmoid functions. By choosing appropriate parameters, the neuron can possess hysteresis property and chaos property simultaneously, and exhibits complex dynamic behavior. Utilizing the two properties, the network, consisting the neurons, can own the ability of global optimization. Complex function optimization problems and combinatorial optimization problems are investigated in this letter by

Acknowledgment

This work was supported by the National Natural Science Foundation of China (10402003).

Xiangdong Liu was born in Hubei, China, in 1971. He received the Ph.D. degree in flight vehicle design from Harbin Institute of Technology, Heilongjiang, China, in 1998. His research interests include nonlinear dynamics of complex systems and chaos synchronization.

References (21)

A. Bevilacqua
A methodological approach to parallel simulated annealing on an SMP system
J. Parallel Distrib. Comput.
(2002)
L. Chen et al.
Chaotic simulated annealing by a neural network model with transient chaos
Neural Networks
(1995)
P. Faure et al.
Is there chaos in the brain? I. Concepts of nonlinear dynamics and methods of investigation
C.R. Acad. Sci. Paris, Sci. de la vie/Life Sci.
(2001)
S. Han et al.
Nonlinear channel blind equalization using hybrid genetic algorithm with simulated annealing
Math. Comput. Model.
(2005)
T. Iwai et al.
Effects of correlation among stored patterns on associative dynamics of chaotic neural network
Physica D: Nonlinear Phenom.
(2005)
A. Lim et al.
A simulated annealing and hill-climbing algorithm for the traveling tournament problem
Eur. J. Oper. Res.
(2006)
J.M. Martinez-Otzeta et al.
Classifier hierarchy learning by means of genetic algorithms
Pattern Recognition Lett.
(2006)
A. Potapov et al.
Robust chaos in neural networks
Phys. Lett. A
(2000)
G. Xia et al.
A binary Hopfield neural network with hysteresis for large crossbar packet switches
Neurocomputing
(2005)
X.S. Xu et al.
A method to improve the transiently chaotic neural network
Neurocomputing
(2005)

There are more references available in the full text version of this article.

Cited by (30)

Secure communication based on the synchronous control of hysteretic chaotic neuron
2017, Neurocomputing
Citation Excerpt :
Chaotic neural network is composed of many chaotic neurons, so it has complex coupling characteristic. Based on the conventional chaotic neural network, hysteretic chaotic neuron and neural network are proposed by changing their activation functions [22,23]. The control of the hysteretic chaotic neural network are further studied [24], and good application effects, such as associative memory and optimal computation are got.
In order to improve the physical realization of the chaotic secure communication, a chaos masking encoding method based on the synchronous control of chaotic neurons with different structure is proposed. The conventional chaotic neuron is set as the transmitting system, and the hysteretic chaotic neuron is set as the receiving system. Hysteretic parameters have creep characteristic which make the hysteretic chaotic neuron have uncertainty. Sliding mode control is used to perform the synchronous control, and fuzzy inference is used to reduce the chattering of the sliding mode control. Simulation results prove the effectiveness of the method.
Optimization of hysteretic chaotic neural network based on fuzzy sliding mode control
2016, Neurocomputing
Citation Excerpt :
In other words, chaos in the neuron or neural network is not controlled by the control theory, and the controllability of chaos in the neuron or neural network is not deeply explored. Likewise, hysteretic chaotic neural network [18,19] can be used to resolve associative memory and function optimization problems by annealing strategy. After decaying the hysteretic parameters and self-feedback weights, hysteretic chaotic neural network becomes the conventional Hopfield neural network.
A hysteretic chaotic neural network with the uncertain term is constructed, and a novel control strategy based on fuzzy sliding mode is proposed to control its dynamic behaviors without changing its structure. The control law contains fuzzy inference term which can reduce the chattering on the sliding mode. The system stability is proved by the Lyapunov theory. The hysteretic chaotic neural network based on the fuzzy sliding mode control can be used to resolve the function optimization problem without destroying chaos generation mechanism and changing the structures of the neural network, which is different to the common annealing optimization strategy. Simulation results prove that the control strategy is a valid method to control the dynamic behaviors of the neural network and resolve the function optimization problem.
Associative memory network and its hardware design
2015, Neurocomputing
Citation Excerpt :
The earlier studies on the associative memory focus mostly on the binary patterns. Many associative networks, such as Hopfiled neural network, bidirectional associative memory (BAM) neural network, chaotic neural network, and so on, are proposed to improve the associative performance [10–14]. As the research developing, it has been recognized that the research on associative memory of multi-valued patterns has very important practical value and theoretical significance.
In order to improve the performance of the conventional associative memory network, a novel associative memory network composed of input layer, computing layer, associative layer and output layer is proposed. An improved Hebb learning rule is designed for the associative network to perform the associative memory of strong correlation and multi-valued sample patterns. The associative memory can be performed by the associative network in only one forward calculation. The hardware circuit of the network can be designed by simple devices to ensure its parallel computation ability and meet the real-time requirement. Simulation results show that the network has better associative performance than conventional associative network in the binary patterns associative memory, it can store and associate strong correlation sample patterns, and it can retrieve the distortion multi-valued sample patterns with 40% noise correctly.
A discrete invasive weed optimization algorithm for solving traveling salesman problem
2015, Neurocomputing
The Traveling Salesman Problem (TSP) is one of the typical NP-hard problems. Efficient algorithms for the TSP have been the focus on academic circles at all times. This article proposes a discrete invasive weed optimization (DIWO) to solve TSP. Firstly, weeds individuals encode positive integer, on the basis that the normal distribution of the IWO does not change, and then calculate the fitness value of the weeds individuals. Secondly, the 3-Opt local search operator is used. Finally, an improved complete 2-Opt (I2Opt) is selected as a second local search operator for solve TSP. A benchmarks problem selected from TSPLIB is used to test the algorithm, and the results show that the DIWO algorithm proposed in this article can achieve to results closed to the theoretical optimal values within a reasonable period of time, and has strong robustness.
Chaos control and associative memory of a time-delay globally coupled neural network using symmetric map
2011, Neurocomputing
Citation Excerpt :
In order to exhibit chaotic dynamics better, Aihara and his collaborators proposed chaotic neural network (CNN) models, which are composed of chaotic neurons derived based on the electrophysiological experiments on squid giant axons [1–6]. Ever since then a lot of chaotic neural network models [7–12] have been successively presented and investigated. There are several kinds of chaotic neural networks according to the mechanism of chaos.
A chaotic neural network called time-delay globally coupled neural network using symmetric map (TDSG) is proposed for information processing applications. Firstly, its rich dynamic behaviors are exhibited and the output stability is demonstrated by using a parameter modulated control method. Secondly, the associative memory of TDSG is investigated by the control method. It is observed that the stable output sequence only contains stored pattern and its reverse pattern and the TDSG finally converges to the stored pattern which has the smallest Hamming distance to the initial patterns with noise. At last, strong information recovery ability of the TDSG is illustrated by comparative experiments.
Optimal matching by the transiently chaotic neural network
2009, Applied Soft Computing Journal
Dynamic programming matching (DPM) is a technique that finds an optimal match between two sequences of feature vectors allowing for stretched and compressed sections of the sequence. The purpose of this study is to formulate the matching problem as an optimization task and carry out this optimization problem by means of a chaotic neural network. The proposed method uses TCNN, a Hopfield neural network with decaying self-feedback, to find the best-matching (i.e., the lowest global distance) path between an input and a template. Experimental results show a very good performance for the proposed algorithm in pattern recognition tasks.

View all citing articles on Scopus

Chunbo Xiu was born in Heilongjiang, China, in 1978. He received the Ph.D. degree in Navigation, Guidance and Control from Beijing Institute of Technology, Beijing, China, in 2005. His research interests include neural networks, system modeling, and chaos control.

View full text

LettersA novel hysteretic chaotic neural network and its applications

Abstract

Introduction

Section snippets

A neuron model

Hysteretic chaotic neural network

Numerical experiments

Conclusion

Acknowledgment

J. Parallel Distrib. Comput.

Neural Networks

C.R. Acad. Sci. Paris, Sci. de la vie/Life Sci.

Math. Comput. Model.

Physica D: Nonlinear Phenom.

Eur. J. Oper. Res.

Pattern Recognition Lett.

Phys. Lett. A

Neurocomputing

Neurocomputing

Letters
A novel hysteretic chaotic neural network and its applications