The H2-control for jump linear systems: cluster observations of the Markov state

doi:10.1016/S0005-1098(01)00210-2

Automatica

Volume 38, Issue 2, February 2002, Pages 343-349

https://doi.org/10.1016/S0005-1098(01)00210-2 Get rights and content

Abstract

The H₂-norm control problem of discrete-time Markov jump linear systems is addressed in this paper when part of, or the total of the Markov states is not accessible to the controller. The non-observed part of the Markov states is grouped in a number of clusters of observations; the case with a single cluster retrieves the situation when no Markov state is observed. The control action is provided in linear feedback form, which is invariant on each cluster, and this restricted complexity setting is adopted, aiming at computable solutions. We explore a recent result by de Oliveira, Bernussou, and Geromel (Systems Control Lett. 37 (1999) 261) involving an LMI characterization to establish a H₂ solution that is stabilizing in the mean square sense. The novelty of the method is that it can handle in LMI form the situation ranging from no Markov state observation to complete state observation. In addition, when the state observation is complete, the optimal H₂-norm solution is retrieved.

Introduction

Markov jump linear systems (MJLS) form a class of processes that presents changes of structure or modes according to an underlying Markov chain. Between jumps of the Markov state, the system evolves according to a linear description associated with the specific Markov state. The ability to model systems subject to abrupt variations in their structures due to failure or repairs, sudden changes in the environmental or in exogenous economic variables, among others, has attracted interest. The theory of stability, optimal LQ control and H_∞ control for MJLS is fairly complete and can be found in several papers spanning for more than one decade.

There are several works dealing with the control with complete state observation (Markov and linear states) for MJLS but fewer involving partial state observations. The information structure that is available for the controller is generally intrinsic to the problem that the MJLS is to model. For instance, if the changes in the Markov chain are associated to failures of components of a non-critical significance, or if they indicate changes between different operating points in a nonlinear plant, or even if some changes are difficult to measure, it is quite possible that the associated Markov states are not accessible to the controller. Although this scenario, from the point of view of applications seems more natural than that of the complete state observation, partial state observations impose serious limitations in the analytical front. This explains in part the considerable development and flourishing of the literature concerning complete state observation; in many situations it can be drawn from the large body of linear system theory, a parallel that is weaker when one deals with partial state observations. Some bridges can be drawn regarding restriction to the linear state observations, e.g. see de Farias, Geromel, do Val, and Costa (2000) or Ji and Chizeck (1992). However, when one regards partial state observations of the Markov state, no simple parallel to the linear system theory is known.

In principle, the analysis can only rely on general stochastic methods, which are able to provide solutions, but rarely in explicit or computable form. One approach to circumvent this difficulty is to adopt linear feedback controls in a restricted complexity setting, and we can mention Caines and Zhang (1995), Costa, do Val, and Geromel (1997), do Val and Basar (1999) and Pan and Bar-Shalom (1996), along this line. In Caines and Zhang (1995) and Pan and Bar-Shalom (1996) the stability of the closed loop system for the problem with no observation of the Markov state is studied, but the analyses do not produce a synthesis of stabilizing controls. In Costa et al. (1997) a solution is attempted but it cannot be posed in LMI form, and in do Val and Basar (1999) the receding horizon control problem is studied with no observation of the Markov state or with observations restricted to clusters of states. The optimal LQ control sequence is determined but stability cannot be assured.

In this paper, we deal with the problem of H₂-norm control of MJLS without observations of the Markov state or observations restricted to clusters of the Markov state, to be defined more precisely in Section 3. We adopt the point of view of restricted complexity synthesis by imposing control actions defined in linear feedback form. We account for an LMI characterization presented in de Oliveira, Bernussou, and Geromel (1999), valid originally to deterministic robust control problems. We adapt it to the MJLS to verify stability in the mean square sense, and to formulate the H₂-norm problem as a convex problem. The result attests that any solution to the convex problem is a stabilizing solution in the mean square sense, and in addition, the optimal H₂ solution is retrieved whenever complete Markov state observation is allowed. This characterization is presented in Theorem 6; the case with no observation of the Markov state is also addressed. Finally, to expose the technique, two examples are presented in Section 4.

Section snippets

Basic formulation and concepts

Let $X ={1,…,N}$ be an index set, and consider the collections of real matrices: A=(A₁,…,A_N), dim(A_i)=n×n, E=(E₁,…,E_N), dim(E_i)=n×m and C=(C₁,…,C_N), dim(C_i)=p×n, i=1,…,N. Let us consider a discrete-time homogeneous Markov chain, $Θ={θ_{k}; k⩾0}$ having $X$ as state space and $P =[p_{ij}]$ , i,j=1,…,N as the transition probability matrix. The probability distribution of the Markov chain at the initial time is given by μ=(μ₁,…,μ_N) in such a way that P(θ₀=i)=μ_i.

Consider a fundamental probability space $(Ω, F,{F_{k}}, P)$ .

The H₂-control problem

The main objective of this paper is to study controlled MJLS with partial observations of the Markov state. Let B=(B₁,…,B_N), dim(B_i)=n×r and D=(D₁,…,D_N), dim(D_i)=q×m for each i=1,…,N, be some associated set of matrices, and consider the stochastic system $G : x_{k} =A(θ_{k})x_{k} +E(θ_{k})w_{k} +B(θ_{k})u_{k}, z_{k} =C(θ_{k})x_{k} +D(θ_{k})u_{k,} k⩾0, w∈ℓ_{2}^{m}, E(|x_{0} |^{2})<∞, θ_{0} ∼μ,$ where $u={u_{k}; k⩾0}$ is the control action. We assume that only part of the states of the Markov chain is accessible, and the control action can rely only on information on

Examples

Example 1

It is drawn from Ji and Chizeck (1990, Example 6.4). The system data are $A_{1} = 2231, B_{1} = 21, E_{1} = 0.5 00 0.4,C_{1} = 1 −1 1100, D_{1} = 001,$ $A_{2} = 10 0.5 1, B_{2} = 00, E_{2} = 100 0.8,C_{2} = 100100, D_{2} = 001,$ $μ=[0 1], P = 0.9 0.1 0.8 0.2 .$ The second mode represents an actuator failure. The results obtained for this example are presented in Table 1. The guaranteed cost solution refers to the deterministic worst case synthesis as in Geromel, Peres, and Souza (1993). The examples in this section were implemented using the LMIsol package (Oliveira, Faria,

Conclusions

In this work a solution to the H₂-control problem of MJLS with incomplete observations of the Markov state is developed in LMI form. The cases with no observation of the Markov state or with observations by clusters, are dealt in the restricted complexity setting, involving linear state feedback controls in an appropriated class.

The solution retrieves the optimal H₂ solution when complete Markov state observation is allowed, and it provides controllers that are stabilizing in the mean square

References (12)

O.L.V. Costa et al.
Stability results for discrete-time linear systems with Markovian jumping parameters
Journal of Mathematical Analysis and Applications
(1993)
M.C. de Oliveira et al.
A new discrete-time robust stability condition
Systems and Control Letters
(1999)
J.B.R. do Val et al.
Receding horizon control of jump linear systems and a macroeconomic policy problem
Journal of Economic Dynamics and Control
(1999)
H. Abou-Kandil et al.
Solution and asymptotic behavior of coupled Riccati equations in jump linear systems
IEEE Transactions on Automatic Control
(1994)
P.E. Caines et al.
On the adaptive control of jump parameter systems via nonlinear filtering
SIAM Journal on Control and Optimization
(1995)
O.L.V. Costa et al.
A convex programming approach to H₂-control of discrete-time Markovian jump linear systems
International Journal of Control
(1997)

There are more references available in the full text version of this article.

Cited by (159)

Learning-based stabilization of Markov jump linear systems
2024, Neurocomputing
In this paper, we explore the stabilization problem of discrete-time Markov jump linear systems from a new perspective. We establish a novel learning-based framework that combines control theory and learning methods to design stabilizing feedback gains. Firstly, we reformulate the stabilization problems for discrete-time Markov jump linear systems into finite-time counterparts. Subsequently, leveraging techniques from the field of learning, we effectively and efficiently solve the finite-time stabilization problems. We systematically investigate two typical stabilization problems of discrete-time Markov jump linear systems within the proposed framework, namely the detector-based feedback stabilization and the static output feedback stabilization problems. Extensive simulation on various numerical examples demonstrates the advantages of our approach over several existing methods for discrete-time Markov jump linear systems.
Energy-to-Peak Static Output Control for Continuous-Time Hidden Markov Jump Linear Systems
2023, IFAC-PapersOnLine
We study the L₂ — L_∞ static output feedback control, also known as energy-to-peak control, for continuous-time Markov jump systems in a context of partial observation of the Markov chain. We consider that the controller has only access to an observed process related to the Markov chain of the plant by means of an exponential hidden Markov model. Sufficient conditions for the design of output feedback controllers satisfying an upper bound on the L₂—L_∞ ratio are given in terms of Linear Matrix Inequalities (LMIs). Based on this result a coordinate descent algorithm is presented to reduce the L₂—L_∞ upper bound ratio, starting from a feasible solution for the LMIs restrictions. The paper is conclude with a numerical example to illustrate the results.
Reduced-order energy-to-peak filtering for hidden Markov jump linear systems
2023, Journal of the Franklin Institute
This paper deals with the design of a energy-to-peak (also known as $ℓ_{2} / ℓ_{\infty}$ ) reduced-order filter for discrete-time Markov jump linear systems assuming that the filter has only access to an estimation of the Markov parameter, coming from the output of a detector device. To model this situation we consider that the process consisting of the Markov chain and the detector signal is a hidden Markov process. The main result shows that, by fixing the filter’s order to $\hat{n}$ , if a set of linear matrix inequalities (LMI) holds true, then we can design a filter of order $\hat{n}$ , which depends only on the estimation of the Markov parameter, such that an upper bound for the $ℓ_{2} / ℓ_{\infty}$ ratio between the output peak norm value and the $ℓ_{2}$ norm of an external disturbance is satisfied. It is also shown that our approach encompasses the case of networked-induced delay systems with imperfect measurement of the delay variable, and the robust polytopic case, with uncertainties on the Markov transition probabilities and detection probabilities. The paper is concluded with an illustrative example using the available computational LMI package tools.
Synchronization for Markovian coupled neural networks with partial mode observation: The finite-time case
2020, Journal of the Franklin Institute
Citation Excerpt :
Therefore, this problem has received much research attention (see, e.g., [33–38]). The H2-norm controllers have been designed for MJSs without state observations or with limited clusters of observations [33]. A detector-based approach that encompasses three types of state information, including complete state information, no state information, and cluster observation information, has been proposed for MJSs [39].
This paper studies the finite-time synchronization for coupled neural networks with jumping inner coupling and non-fragile controller. A hidden-Markov observation model that covers many relevant Markovian jumping cases is applied to describe a more general situation where the synchronization controller has limited capability and partial modes observations. In terms of the hidden-Markov observations, a sufficient condition on the finite-time synchronization boundness is firstly derived by constructing a mode-dependent Lyapunov function for the augmented synchronization error system. Then an LMI framework is proposed for designing non-fragile synchronization controller gains. Finally, the effectiveness and the validity are demonstrated by a numerical example.
Mixed H<inf>2</inf>/H<inf>∞</inf> state-feedback control of continuous-time Markov jump systems with partial observations of the Markov chain
2020, IFAC-PapersOnLine
We study the mixed ℋ₂/ℋ_∞ state-feedback control of continuous-time Markov jump linear systems considering that the Markov chain is not observable, with the only information available to the controller coming from the output of a fault-detection and isolation device. We present sufficient design conditions given in terms of linear matrix inequalities so that the closed-loop system is stable and its ℋ₂ and ℋ_∞ norms are bounded. We present an illustrative example in which we investigate the behavior of the proposed algorithm.
Robust H<inf>2</inf> filtering of phase-type semi-Markov jump linear systems with cluster observations
2024, International Journal of Robust and Nonlinear Control

View all citing articles on Scopus

João B.R. do Val was born in São Paulo State, Brazil in 1955. He received the B.S. and M.S. degrees in Electrical Engineering from the University of Campinas (UNICAMP), Campinas, Brazil in 1977 and 1981, respectively and the Ph.D. degree in Electrical Engineering from the Imperial College of Science and Technology in London in 1985. He also received the Diploma of Imperial College in 1985. Since 1978 he has held a position at the Faculty of Electrical Engineering of UNICAMP. During the years 1996 and 1997 he was a Visiting Scholar at the Decision and Control Laboratory (Coordinated Science Laboratory) at University of Illinois at Urbana-Champaign. Since 2000, he is the editor of the journal “Controle Automaçâo” published by the Brazilian Society of Automatica. His research interests include Stochastic Systems and Control, Jump Processes, with applications in Operation Research and Communication problems.

José C. Geromel was born in Itatiba, Brazil in 1952. He received the B.S. and M.S. degrees in Electrical Engineering from the University of Campinas (UNICAMP), Campinas, Brazil, in 1975 and 1976, respectively. He received the Docteur d’ Etat degree from the University Paul Sabatier, Toulouse, France, in 1979. In 1975, he joined the School of Electrical Engineering, UNICAMP, where he is presently Professor of Control Theory. In 1987, he was a Visiting Professor at Milan Polytecnique Institute, Milan, Italy. His current research interests include convex programming theory, robust control systems design, robust filtering and joint location and control systems design. Prof. Geromel is a member of the Editorial Board of Studies in Informatics and Control and Subject Editor of the International Journal of Robust and Nonlinear Control. In 1994, he was awarded the Zeferino Vaz Award for his teaching and research activities at UNICAMP. Since 1991, he has been a Fellow of the CNPq — the Brazilian Council for Research and Development. In 1999, he has been named for Chevalier dans l’ Ordre des Palmes Academiques by the Minister of National Education of France. Since 1998, he is the Dean for Graduate Studies at UNICAMP and member of the Brazilian Academy of Science. He is co-author of the book Control Theory and Design: An RH₂and RH_∞Viewpoint (with P. Colaneri and A. Locatelli, New York, Academic Press, 1997).

Alim P.C. Gonçalves was an undergraduate student at the School of Electrical and Computing Engineering, UNICAMP from 1995 to 2000, when he received his B.S. degree in Electrical Engineering. He was awarded a scholarship in an undergraduate program, to spend one year at the University of Stuttgart, Germany. He also received a scholarship during his studies on Jump Markov Systems. Presently, he holds a position at Lucent Technologies in Brazil.

^☆: This paper was not presented at any IFAC meeting. This paper was recommended for publication in revised form by Associate Editor R. Srikant under the direction of Editor Tamer Basar. Research supported in part by CNPq, Grant 300721/86-2(RN) and the PRONEX Grant 015/98 ‘Control of Dynamical Systems’.

View full text

Brief PaperThe H2-control for jump linear systems: cluster observations of the Markov state☆

Abstract

Introduction

Section snippets

Basic formulation and concepts

The H2-control problem

Examples

Conclusions

Journal of Mathematical Analysis and Applications

Systems and Control Letters

Journal of Economic Dynamics and Control

Solution and asymptotic behavior of coupled Riccati equations in jump linear systems

IEEE Transactions on Automatic Control

On the adaptive control of jump parameter systems via nonlinear filtering

SIAM Journal on Control and Optimization

A convex programming approach to H2-control of discrete-time Markovian jump linear systems

International Journal of Control

Brief Paper
The H₂-control for jump linear systems: cluster observations of the Markov state☆

The H₂-control problem

A convex programming approach to H₂-control of discrete-time Markovian jump linear systems