Exponential stability of delayed recurrent neural networks with Markovian jumping parameters

doi:10.1016/j.physleta.2006.03.078

Physics Letters A

Volume 356, Issues 4–5, 14 August 2006, Pages 346-352

https://doi.org/10.1016/j.physleta.2006.03.078 Get rights and content

Abstract

In this Letter, the global exponential stability analysis problem is considered for a class of recurrent neural networks (RNNs) with time delays and Markovian jumping parameters. The jumping parameters considered here are generated from a continuous-time discrete-state homogeneous Markov process, which are governed by a Markov process with discrete and finite state space. The purpose of the problem addressed is to derive some easy-to-test conditions such that the dynamics of the neural network is stochastically exponentially stable in the mean square, independent of the time delay. By employing a new Lyapunov–Krasovskii functional, a linear matrix inequality (LMI) approach is developed to establish the desired sufficient conditions, and therefore the global exponential stability in the mean square for the delayed RNNs can be easily checked by utilizing the numerically efficient Matlab LMI toolbox, and no tuning of parameters is required. A numerical example is exploited to show the usefulness of the derived LMI-based stability conditions.

Introduction

In the past few decades, the mathematical properties of the recurrent neural networks (RNNs), such as the stability, the attractivity and the oscillation, have been intensively investigated. RNNs have been successfully applied in many areas, including image processing, pattern recognition, associative memory, and optimization problems. In fact, the stability analysis issue for RNNs with time delays has been an attractive subject of research in the past few years, where the time delays under consideration can be classified as constant delays, time-varying delays, and distributed delays. Various sufficient conditions, either delay-dependent or delay-independent, have been proposed to guarantee the global asymptotic or exponential stability for the RNNs with time-delays, see e.g. [1], [2], [3], [4] for some recent publications, where many methods have been exploited, such as the LMI approach and M-matrix approach.

Traditional RNNs assume that the continuous variables propagate from one processing unit to the next. However, RNNs sometimes have problems in catching long-term dependencies in the input stream. As the temporal sequences increase in length, the influence of early components of the sequence has less impact on the network output. Such a phenomenon is referred to as the problem of information latching [5]. A widely used approach to dealing with the information latching problem is to extract finite state representations (also called clusters, patterns, or modes) from trained networks [6], [7], [8], [9]. In other words, the RNNs may have finite modes, and the modes may switch (or jump) from one to another at different times. Recently, it has been shown in [10] that, the switching (or jumping) between different RNN modes can be governed by a Markovian chain. Hence, an RNN with such a “jumping” character may be modeled as a hybrid one; that is, the state space of the RNN contains both discrete and continuous states. For a specific mode, the dynamics of the RNN is continuous, but the parameter jumps among different modes may be seen as discrete events. Note that the concept of Markovian neural networks has already been used in some papers, see e.g. [11]. Therefore, RNNs with Markovian jumping parameters are of great significance in modeling a class of neural networks with finite network modes. It should be pointed out that, up to now, the stability analysis problem for RNNs with Markovian switching has received very little research attention, despite its practical importance.

In this Letter, we are concerned with the analysis issue for the global exponential stability of RNNs with mixed time-delays and Markovian jumping parameters. To the best of the authors' knowledge, this is the first attempt to introduce and investigate the delayed RNNs with Markovian switching. It is worth mentioning that the control and filtering problems for Markovian jump systems (MJSs) have already been widely studied in the control and signal processing communities, see e.g. [12], [13]. The main purpose of this Letter is to establish LMI-based stability criteria for testing whether the network dynamics is stochastically exponentially stable in the mean square, independent of the time delay. It is known that LMIs can be efficiently solved by utilizing the numerically attractive Matlab LMI toolbox, hence our proposed results would be practical. We will use a simple example to illustrate the usefulness of the derived LMI-based stability conditions.

Section snippets

Problem formulation

Notation

The notations in this Letter are quite standard. $R^{n}$ and $R^{n \times m}$ denote, respectively, the n-dimensional Euclidean space and the set of all $n \times m$ real matrices. The superscript “T” denotes the transpose and the notation $X ⩾ Y$ (respectively, $X > Y$ ) where X and Y are symmetric matrices, means that $X - Y$ is positive semi-definite (respectively, positive definite). I is the identity matrix with compatible dimension. We let $h > 0$ and $C ([- h, 0]; R^{n})$ denote the family of continuous functions φ from $[- h, 0]$ to $R^{n}$ with

Main results and proofs

Let us first give the following lemmas which will be frequently used in the proofs of our main results in this Letter.

Lemma 1

Let $x \in R^{n}$ , $y \in R^{n}$ and $ε > 0$ . Then we have $x^{T} y + y^{T} x ⩽ ε x^{T} x + ε^{−1} y^{T} y$ .

Proof

The proof follows from the inequality ${(ε^{1 / 2} x - ε^{- 1 / 2} y)}^{T} (ε^{1 / 2} x - ε^{- 1 / 2} y) ⩾ 0$ immediately. □

Lemma 2

[14] Given constant matrices $Ω_{1}$ , $Ω_{2}$ , $Ω_{3}$ where $Ω_{1} = Ω_{1}^{T}$ and $0 < Ω_{2} = Ω_{2}^{T}$ , then $Ω_{1} + Ω_{3}^{T} Ω_{2}^{−1} Ω_{3} < 0$ if and only if $[\begin{matrix} Ω_{1} & Ω_{3}^{T} \\ Ω_{3} & - Ω_{2} \end{matrix}] < 0, or [\begin{matrix} - Ω_{2} & Ω_{3} \\ Ω_{3}^{T} & Ω_{1} \end{matrix}] < 0 .$

The main results of this Letter are given as follows, which shows that the network dynamics of (5) is

A numerical example

We present a simple example here in order to illustrate the usefulness of our main results. Our aim is to examine the global exponential stability of a given delayed neural network with Markovian jumping parameters.

Consider a two-neuron delayed neural network (5) with two modes. The network parameters are given as follows: $A_{1} = [\begin{matrix} 1.6 & 0 \\ 0 & 1.8 \end{matrix}], A_{2} = [\begin{matrix} 1.2 & 0 \\ 0 & 1.5 \end{matrix}], G_{0} = [\begin{matrix} 0.2 & 0 \\ 0 & 0.3 \end{matrix}], G_{1} = [\begin{matrix} 0.4 & 0 \\ 0 & 0.6 \end{matrix}], W_{01} = [\begin{matrix} 1.2 & - 1.5 \\ - 1.7 & 1.2 \end{matrix}],$ $W_{02} = [\begin{matrix} 0.6 & 0.1 \\ 0.1 & 0.2 \end{matrix}], W_{11} = [\begin{matrix} 1.1 & 0.5 \\ 0.5 & 0.8 \end{matrix}], W_{12} = [\begin{matrix} 0.8 & 0.2 \\ 0.2 & 0.3 \end{matrix}], Γ = [\begin{matrix} - 7 & 7 \\ 6 & - 6 \end{matrix}] .$

By using the Matlab LMI

Conclusions

In this Letter, we have dealt with the problem of global exponential stability analysis for a class of general recurrent neural networks, which both time delays and Markovian jumping parameters. We have removed the traditional monotonicity and smoothness assumptions on the activation function. A linear matrix inequality (LMI) approach has been developed to solve the problem addressed. The conditions for the global exponential stability have been derived in terms of the positive definite

References (19)

J. Cao et al.
Chaos Solitons Fractals
(2005)
H. Huang et al.
Appl. Math. Comput.
(2003)
J. Liang et al.
Phys. Lett. A
(2003)
Z. Wang et al.
Phys. Lett. A
(2005)
J.L. Elman
Cognitive Sci.
(1990)
M. Kovacic
Eur. J. Oper. Res.
(1993)
Y. Bengio, P. Frasconi, P. Simard, The problem of learning long-term dependencies in recurrent networks, in:...
D. Bolle et al.
J. Phys. A
(1992)
M.P. Casey
Neural Comput.
(1996)

There are more references available in the full text version of this article.

Cited by (357)

A novel formulation of RNN-based neural network with real-time updating – An application for dynamic hydraulic fractured shale gas production forecasting
2024, Geoenergy Science and Engineering
The Recurrent Neural Network (RNN) has found extensive application in production forecasting for fractured reservoirs, particularly excelling at accurately predicting self-regression in late-life prediction for wells with substantial historical data. However, forecasting for new wells or those with limited historical data presents greater challenges. The main difference lies in the insufficiency of historical data, which requires meaningful integration of a continuous production data stream to enhance prediction results. In the context of utilizing real-time data and continuously updating predictions, many existing formulations of RNN-based models in the literature either utilize inputs of sub-sequences instead of the entire sequence or require retraining of the entire model to handle new data and update the predictions. Therefore, we propose a novel prediction framework called Recurrent Updated Forecasting (RUF). This framework considers the entire time series structure, improves accuracy and training efficiency, and allows for prediction updates without retraining. In scenarios without production data, we employ a Radial Basis Function Network (RBFN) to initialize the process. Once more extended production history becomes available, it can be utilized to enhance prediction results. We present a case study based on both synthetic and field data from the Montney shale gas reservoir, considering different production-related features as inputs to predict gas production rates over a three-year period. The proposed method is evaluated using the P10, P50, and P90 values of the testing Root Mean Square Error (RMSE). The results demonstrate that our method outperforms existing approaches, exhibiting outstanding efficiency, reliability, and accuracy.
New Criteria of Event-triggered Exponential State Estimation for Delayed semi-Markovian Memristor-based Neural Networks
2023, Expert Systems with Applications
This paper focuses on the exponential state estimation issue for Memristor-based Neural Networks (MNNs) with semi-Markov Jump Parameters (semi-MJPs) and Time-varying Delay (TD). At first, an alternative way obtains a tighter bound of the Exponential-type Integral Quadratic Terms (EIQTs) is introduced by establishing a novel Exponential-type Free-matrix Integral Inequality (EFMII), which contains some existing integral inequalities as its special cases. Next, a Sampled-data-based Event-triggered Control Mechanism (SECM) is introduced to further save the limited network resources. Moreover, a proper Lyapunov-Krasovskii Functional (LKF) is structured by getting the utmost out of the information of decay exponent, semi-MJPs, and TD. Then, by using the EFMII and SECM, three new kinds of less conservative criteria ensuring the exponential stability of the underlying error systems are found in terms of Linear Matrix Inequalities (LMIs), and the corresponding state estimators are well designed. At last, the validity of the proposed theoretical results is shown by three numerical simulation examples.
Extended dissipativity-based synchronization of Markov jump neural networks subject to partially known transition and mode detection information
2023, Neurocomputing
This paper addresses the extended dissipativity-based synchronization problem of Markov jump neural networks with partially known probability information by using a detector from the hidden Markov model, where the partially known probability may exist in one of the transition probability matrix and detection probability matrix, or both of them simultaneously. By using such a hidden Markov model, an extended stochastic dissipative synchronization criterion for neural networks with partially known probability information is established, and a novel design method is given with the help of an improved activation function dividing method. Finally, the validity of the proposed approach is demonstrated by two numerical examples.
H<inf>∞</inf> performance analysis for delayed Markovian jump neural networks via the Lyapunov-Krasovskii functional with delay-product-type terms
2021, Journal of the Franklin Institute
This paper studies the problem of $H_{\infty}$ performance analysis for delayed Markovian jump neural networks (MJNNs). Firstly, the third-order Bessel–Legendre (B-L) integral inequality based on reciprocally convex approach is derived, which contains some existing ones as its special cases. Secondly, in order to facilitate the use of more information about time delays, the Lyapunov–Krasovskii functional (LKF) with delay-product-type terms is proposed. As a result, by utilizing the LKF with delay-product-type terms and the third-order B-L integral inequality based on reciprocally convex approach, less conservative sufficient conditions that guarantee the $H_{\infty}$ performance for delayed MJNNs are obtained. Finally, simulation results are given to illustrate the effectiveness of the proposed method.
State estimation of Markov jump neural networks with random delays by redundant channels
2021, Neurocomputing
This paper deals with the problem of state estimation for Markov jump neural networks with random delays through redundant channels. First, a redundant channel is provided to increase the probability of successfully transmitting the measurements over the shared communication network, and two mutually independent Bernoulli sequences are used to reflect the data dropouts phenomena in the main channel and the redundant channel. In addition, the time-varying random delay is both mode-dependent and distribution-dependent, and its probabilistic characteristic obeys Bernoulli distribution. Then, by constructing a suitable Lyapunov-Krasovskii functional, the sufficient condition is obtained to guarantee the augmented estimation error system is mean-square stable with a prescribed $H_{\infty}$ disturbance attenuation performance. Meanwhile, a mode-dependent estimator is designed by convex optimization method. Finally, the effectiveness of the proposed method is demonstrated by a simulation example.
Asynchronous finite-time state estimation for semi-Markovian jump neural networks with randomly occurred sensor nonlinearities
2021, Neurocomputing
This paper addresses the finite-time state estimation problem for semi-Markovian jump neural networks with sensor nonlinearities under the consideration of leakage delay and time-varying delay. The modes of original system and desired estimator are supposed to be asynchronous. Some sufficient conditions are proposed to guarantee the finite-time boundedness as well as mixed $H_{\infty}$ and passive performance of the error system in terms of constructing Lyapunov–Krasovskii functionals. By utilizing affine Bessel-Legendre inequalities, a less conservative result can be achieved. By virtue of linear matrices inequalities approach, the asynchronous state estimator gains are obtained. Two numerical examples are provided to demonstrate the less conservativeness and effectiveness of our method.

View all citing articles on Scopus

^☆: This work was supported in part by the Engineering and Physical Sciences Research Council (EPSRC) of the UK under Grant GR/S27658/01, the Nuffield Foundation of the UK under Grant NAL/00630/G, and the Alexander von Humboldt Foundation of Germany.

View full text

Exponential stability of delayed recurrent neural networks with Markovian jumping parameters☆

Abstract

Introduction

Section snippets

Problem formulation

Main results and proofs

A numerical example

Conclusions

Chaos Solitons Fractals

Appl. Math. Comput.

Phys. Lett. A

Phys. Lett. A

Cognitive Sci.

Eur. J. Oper. Res.

J. Phys. A

Neural Comput.