EXPONENTIAL STABILITY IN MEAN SQUARE OF DISCRETE-TIME TIME-VARYING LINEAR STOCHASTIC SYSTEMS WITH MARKOVIAN SWITCHING
Vasile Dr˘agan, Toader Morozan
”Simion Stoilow” Institute of Mathematics of the Romanian Academy, Bucharest [email protected], [email protected]
Abstract The problem of the mean square exponential stability for a class of discrete- time time-varying linear stochastic systems subject to Markovian switching is investigated. The case of the linear systems whose coefficients depend both on present state and the previous state of the Markov chain is considered. Three different definitions of the concept of exponential stability in mean square are introduced and it is shown that they are not always equivalent. One definition of the concept of mean square exponential stability is done in terms of the exponential stability of the evolution defined by a sequence of linear positive operators on an ordered Hilbert space. The other two definitions are given in terms of different types of exponential behavior of the trajectories of the considered system. In our approach the Markov chain is not prefixed. The only available information about the Markov chain is the sequence of probability transition matrices and the set of its states. In this way one obtains that if the system is affected by Markovian jumping, the property of exponential stability is independent of the initial distribution of the Markov chain.
The definition expressed in terms of exponential stability of the evolution generated by a sequence of linear positive operators, allows us to characterize the mean square exponential stability based on the existence of some quadratic Lyapunov functions.
77
The results developed in this paper may be used to derive some procedures for designing stabilizing controllers for the considered class of discrete-time linear stochastic systems in the presence of a delay in the data transmission.
Keywords: discrete-time linear stochastic systems, Markov chains, mean square exponential stability, Lyapunov operators, delays in data transmission.
1. INTRODUCTION
Stability is one of the main tasks in the analysis and synthesis of a controller in many control problems such as: linear quadratic regulator, H2 optimal control, H∞-control and other robust control problems (see e.g. [9, 16, 24]
for the continuous time case or [1, 4, 5, 15] for the discrete-time case and references therein).
For linear stochastic systems there are various types of stability. However one of the most popular among them is ”exponential stability in mean square” (ESMS). This is due to the existence of some efficient algorithms to check this property.
In the literature, the problem of ESMS of discrete-time linear stochastic systems with Markovian jumping was intensively investigated. For the readers convenience we refer to [2, 3, 12, 13, 14, 15, 17, 18, 19, 20, 21, 22]. The majority of the works where the problem of ESMS of discrete-time linear systems with Markovian switching is investigated, deals with the so-called time invariant case. That is the case of discrete-time linear systems with the matrix coefficients not depending on time and the Markov chain being homogeneous.
There are papers [19, 21, 22] where the matrix coefficients of the system are time dependent and the Markov chain is homogeneous. There are also papers [12, 17] where the matrix coefficients of the discrete time system do not depend on time but the Markov chain is inhomogeneous.
In [8], it was considered the general situation of the discrete-time time- varying linear stochastic systems corrupted by independent random perturba- tions and by jump Markov perturbations. The general case of discrete-time time-varying linear systems with Markovian switching was investigated in [10].
The aim of the present paper is to extend the results of [10] to the case of discrete-time time-varying linear stochastic systems whose coefficients depend both to the present state ηtand the previous stateηt−1 of the Markov chain.
We assume that the matrix coefficients may depend on time and the Markov chain is not necessarily homogeneous. Such systems arise in connection with the problem of designing of a stabilizing feedback gain in the presence of some delay in transmission of the date either on the channel from the sensors to controller or between controller and actuators. For this class of systems we introduce three different definitions of the concept of exponential stability in mean square. One of these definitions characterizes the concept of exponential stability in mean square in terms of exponential stability of the evolution generated by a suitable sequence of linear positive operators associated with the considered stochastic system. This type of exponential stability in mean square which we shall called ”strong exponential stability in mean square”
(SESMS) is equivalent to the existence of a quadratic Lyapunov function;
meaning that it is equivalent to the solvability of some systems of linear matrix equations or linear matrix inequations.
Other two types of exponential stability in mean square are stated in terms of exponential behavior of the state space trajectories of the considered sto- chastic systems.
We show that the three definitions of the exponential stability in mean square are not always equivalent. Also, we prove that under some additional assumptions part of types of exponential stability in mean square become equivalent. In the case of discrete-time linear stochastic systems with peri-
odic coefficients all these three types of exponential stability in mean square introduced in the paper become equivalent.
We remark that in our approach the Markov chain is not prefixed. The only available information about the Markov chain consists of the sequence of transition probability matrices {Pt}t≥0 and the set D of its states. The initial distributions of the Markov chain do not play any role in defining and characterizing the exponential stability in mean square.
2. THE PROBLEM
2.1. DESCRIPTION OF THE SYSTEMS
Let us consider discrete-time linear stochastic systems of the form
x(t+ 1) =A(t, ηt, ηt−1)x(t) (1) t≥1,t∈Z+, where x∈Rn and{ηt}t≥0 is a Markov chain defined on a given probability space (Ω,F,P) with the state space the finite setD={1,2, ..., N} and the sequence of transition probability matrices{Pt}t≥0. This means that fort≥0,Ptare stochastic matrices of size N, with the property
P{ηt+1 =j|Gt}=pt(ηt, j) (2) for all j ∈ D, t ≥ 0, t ∈ Z+, where Gt = σ[η0, η1, ..., ηt] is the σ-algebra generated by the random variables ηs,0≤s≤t. For more details concerning the properties of Markov chains and of the sequences {Pt}t≥0 of stochastic matrices we refer to [6].
IfPt=P for allt≥0 then the Markov chain is known as ahomogeneous Markov chain.
In this paper we investigate several aspects of the issue of exponential sta- bility in mean square of the solution x = 0 of the system (1). Our aim is to reveal some difficulties which are due to the fact that the coefficients of the
system are time varying andηtis an inhomogeneous Markov chain. In order to motivate the study of the problem of exponential stability in mean square for the case of stochastic systems of type (1) we consider the following controlled system
x(t+ 1) =A(t, ηt)x(t) +B(t, ηt)u(t)y(t) =C(t, ηt)x(t), t≥0, (3) where x(t) ∈ Rn is the state vector, u(t) ∈ Rm is the vector of the control inputs and y(t)∈Rp is the vector of the measurements. The aim is to design a control law of the form
u(t) =K(t, ηt)y(t) (4)
with the property that the trajectories of the corresponding closed-loop system satisfy
tlim→∞E[|x(t)|2] = 0. (5) In the case when on the channel from the sensor to the controller there exists a delay in transmission of the measurement then, the control (4) is replaced by
u(t) =K(t, ηt)y(t−1). (6)
If the delay occurs on the channel from controller to actuators then instead of (4) we will have
u(t) =K(t−1, ηt−1)y(t−1). (7)
We have to solve the following two problems:
P1: Find a set of sufficient conditions which guarantee the existence of the matrices K(t, i)∈Rm×p, i∈D, t≥0, such that the trajectories of the closed- loop system obtained by coupling (6) with (3) will satisfy a condition of type (5).
P2: Find a set of conditions which guarantee the existence of the matrices K(t, i)∈Rm×p with the property that the trajectories of the closed-loop system
obtained by coupling the control (7) with the system (3) satisfy a condition of type (5).
Coupling a control (6) to the system (3) one obtains the following closed- loop system
x(t+ 1) =A(t, ηt)x(t) +B(t, ηt)K(t, ηt)C(t−1, ηt−1)x(t−1). (8) The closed-loop system obtained combining (7) to (3) is
x(t+ 1) =A(t, ηt)x(t) +B(t, ηt)K(t−1, ηt−1)C(t−1, ηt−1)x(t−1). (9) Setting ξ(t) = (xT(t), xT(t−1))T we obtain the following version of (8) and (9)
ξ(t+ 1) = ˜A(t, ηt, ηt−1)ξ(t), t≥1, (10) ξ(t+ 1) = ˆA(t, ηt, ηt−1)ξ(t), t≥1, (11) respectively, where
A(t, i, j) =˜
⎛
⎝ A(t, i) B(t, i)K(t, i)C(t−1, j)
In 0
⎞
⎠,
A(t, i, j) =ˆ
⎛
⎝ A(t, i) B(t, i)K(t−1, j)C(t−1, j)
In 0
⎞
⎠. (12) Hence, the system (10) as well as (11) are both of the form (1).
The condition (5) is equivalent to
tlim→∞E[|ξ(t)|2] = 0. (13) This motivates the consideration of the problem of stability in mean square of systems of type (1).
Setπt= (πt(1), πt(2), ..., πt(N)) the distribution of the random variableηt, that is πt(i) =P{ηt=i}. It can be verified that the sequence {πt}t≥0 solves
πt+1 =πtPt, t≥0. (14)
Remark 2.1. a) From (14) it follows that it is possible that πt(i) = 0 for some t ≥ 1, i ∈ D even if π0(j) > 0, 1 ≤ j ≤ N. This is specific for the discrete-time case.
b) The only available information concerning the system (1) is the setDand the sequences {Pt}t∈Z+, {A(t, i, j)}t≥1, i, j ∈ D. The initial distributions of the Markov chain are not prefixed. Hence, throughout the paper by a Markov chain we will understand any triple{{ηt}t≥0, {Pt}t≥0, D} whereDis a fixed set D= {1,2, ..., N}, {Pt}t≥0 is a given sequence of stochastic matrices and {ηt}t≥0 is an arbitrary sequence of random variables taking values in Dand satisfying (2).
Remark 2.1 a) allows us to define the following subsets of the setD Ds={i∈D|πs(i)>0} (15) for each integers≥0.
Define Θ(t, s) as follows: Θ(t, s) =A(t−1)A(t−2)...A(s) ift≥s+ 1 and Θ(t, s) =In ift=s, s≥1.
Any solution x(t) of (1) verifies
x(t) = Θ(t, s)x(s)
Θ(t, s) will be called the fundamental (random) matrix solution of (1).
2.2. LYAPUNOV TYPE OPERATORS
Let Sn ⊂ Rn×n be the linear subspace of symmetric matrices. Set SNn = Sn⊕Sn⊕...⊕Sn. One can see that SNn is a real ordered Hilbert space (see [7], Example 2.5(iii)). The usual inner product onSNn is
X, Y= N
i=1
T r(Y(i)X(i)) (16)
for all X= (X(1), ..., X(N)) andY = (Y(1), ..., Y(N)) from SNn.
Consider the sequences{A(t, i, j)}t≥1, A(t, i, j)∈Rn×n,i, j∈D,{Pt}t≥0, Pt= (pt(i, j))∈RN×N.
Based on these sequences we construct the linear operators Υt:SNn →SNn, ΥtS = (ΥtS(1), ...,ΥtS(N)) with
ΥtS(i) = N j=1
pt−1(j, i)A(t, i, j)S(j)AT(t, i, j), (17) t≥1, S ∈SNn.
IfA(t, i, j), pt(i, j) are related to the system (1), then the operators Υt are called the Lyapunov type operators associated with the system (1).
By direct calculation based on the definition of the adjoint operator with respect to the inner product (16) one obtains that Υ∗tS= (Υ∗tS(1), ...,Υ∗tS(N)) with
Υ∗tS(i) = N j=1
pt−1(i, j)AT(t, j, i)S(j)A(t, j, i) (18) t≥1, S ∈SNn.
Let R(t, s) be the linear evolution operator defined on SNn by the sequence {Υt}t≥1. Hence R(t, s) = Υt−1Υt−2...Υs if t ≥ s+ 1 and R(t, s) = ISNn if t=s≥1.
IfX(t) is a solution of discrete-time linear equation on SNn
Xt+1 = ΥtXt, (19)
thenXt=R(t, s)Xs for all t≥s≥1.
The following result provides a relationship between the operators R∗(t, s) and the fundamental matrix solution Θ(t, s) of the system (1).
Theorem 2.2. Under the considered assumptions the following equality
(R∗(t, s)H)(i) =E[ΘT(t, s)H(ηt−1)Θ(t, s)|ηs−1 =i] (20) holds for allH = (H(1), ..., H(N))∈SNn, t≥s≥1, i∈Ds−1.
Proof: see [8, 11].
2.3. DEFINITIONS OF MEAN SQUARE EXPONENTIAL STABILITY
Now we are in position to state the concept of exponential stability in mean square of the zero state equilibrium of the system (1).
In this subsection we introduce three different definitions of the concept of exponential stability in mean square.
Definition 2.3. We say that the zero state equilibrium of the system (1) is strongly exponentially stable in mean square (SESMS) if there exist β ≥1, q∈(0,1) such that
R(t, s) ≤βqt−s (21)
for all t≥s≥1.
SinceSNn is a finite dimensional linear space, in (21) one can take any norm inB(SNn).
Definition 2.4. We say that the zero state equilibrium of the system (1) is exponentially stable in mean square with conditioning(ESMS-C) if there exist β ≥1, q∈(0,1) such that for any Markov chain ({ηt}t≥0,{Pt}t≥0,D) we have E[|Θ(t, s)x|2|ηs−1 =i]≤βqt−s|x|2 (22) for all t≥s≥1, x∈Rn, i∈Ds−1.
Definition 2.5. We say that the zero state equilibrium of the system (1) is exponentially stable in mean square (ESMS) if there exist β ≥1, q∈(0,1) such that for any Markov chain ({ηt}t≥0,{Pt}t≥0,D) we have
E[|Θ(t, s)x|2]≤βqt−s|x|2 (23) for all t≥s≥1, x∈Rn.
3. EXPONENTIAL STABILITY IN MEAN SQUARE
In this section we establish the relations between the concepts of exponential stability in mean square introduced in Definition 2.3 -Definition 2.5. Firstly we shall show that in the general case of the time-varying system (1) these definitions are not, in general, equivalent. Finally we show that in the case of systems (1) with periodic coefficients these definitions become equivalent.
3.1. THE GENERAL CASE
On the spaceSNn one introduces the norm| · |1 by
|X|1 =maxi∈D|X(i)| (24) where|X(i)|is the spectral norm of the matrixX(i). Together with the norm
| · |2 induced by the inner product (16), the norm | · |1 will play an important role in the characterization of the strong exponential stability (SESMS).
IfT :SNn →SNn is a linear operator thenTkis the operator norm induced by | · |k, k= 1,2.
We recall that (see Proposition 2.2 in [7]) if T is a linear and positive operator on SNn, then
T1=|T J|1 (25) whereJ = (In, ..., In)∈SNn.
Since R(t, s) and R∗(t, s)1 are equivalent, from (21) it follows that R∗(t+ 1, t)1 ≤β1, t≥0, hence by using (25) and (18) one obtains
Corollary 3.1. If the zero state equilibrium of the system (1) is SESMS, then
{"
pt−1(i, j)A(t, j, i)}t≥1, i, j ∈D,are bounded sequences.
Now we prove
Theorem 3.2. Under the considered assumptions we have:
(i)if the zero state equilibrium of the system (1) is SESMS then it is ESMS- C;
(ii) if the zero state equilibrium of the system (1) is ESMS-C then it is ESMS.
The proof of (i) follows immediately from (20). (ii) follows from the in- equality
E[|Θ(t, s)x|2]≤
i∈Ds−1
E[|Θ(t, s)x|2|ηs−1 =i].
As we can see in Example 3.5 and Example 3.6 from below the validity of the converse implications from the above theorem is not true in the absence of some additional assumptions.
Definition 3.3. We say that a stochastic matrix Pt ∈ RN×N is a non- degenerate stochastic matrix if for any j ∈ D there exists i ∈ D such that pt(i, j)>0.
We remark that if Pt, t ≥ 0 are nondegenerate stochastic matrices, then from (14) it follows that πt(i)>0, t≥1, i∈Difπ0(i)>0 for alli∈D.
Now we have
Theorem 3.4. If for all t ≥0, Pt are nondegenerate stochastic matrices, then the following are equivalent
(i) the zero state equilibrium of the system (1) is SESMS, (ii) the zero state equilibrium of the system (1) is ESMS-C,
(iii)there exist a Markov chain({ηt}t≥0,{Pt}t≥0,D)withP(η0=i)>0, i∈ D such that
E[|Θ(t, s)x|2|ηs−1 =i]≤βqt−s|x|2 (26) for all t≥s≥1, i∈D, x∈Rn where β≥1, q∈(0,1).
Proof (i) → (ii) follows from Theorem 3.2 and (ii) → (iii) is obvious. It remains to prove implication (iii)→(i). Applying Theorem 2.2 for H =J =
(In, ..., In) from (26) one obtains that xT0[R∗(t, s)J](i)x0 ≤βqt−s|x0|2 for all t≥s≥1, i∈D, x0∈Rn.
This allows us to conclude that|[R∗(t, s)J](i)| ≤βqt−s for allt≥s≥1, i∈ D. Based on (24) one gets |R∗(t, s)J|1 ≤ βqt−s for all t ≥ s. Finally from (25) it follows thatR∗(t, s)1≤βq(t−s) for allt≥s≥1. Thus the proof is complete, since R(t, s) and R(t, s)∗1 are equivalent.
The next two examples show that the converse implication in Theorem 3.2 are not always true.
Example 3.5. Consider the system (1) in the particular casen= 1, N = 2 described by
x(t+ 1) =a(t, ηt, ηt−1)x(t) (27) where t ≥ 1, a(t, i,1) = 0, a(t, i,2) = 2t−12 , t ≥ 1, i ∈ {1,2}. The transition probability matrix is
Pt=
⎛
⎝ 1− 4t1+1 1 4t+1
1− 4t1+1 1 4t+1
⎞
⎠, t≥0. (28)
We have
"
pt−1(2,1)a(t,1,2) = (1− 1
4t)122t−12 . Hence the sequence{"
pt−1(2,1)a(t,1,2)}t≥1 is unbounded. Then via Corol- lary 3.1, we deduce that the zero state equilibrium of the system (27) is not SESMS. On the other hand one sees that the transition probability matrix (28) is a non-degenerate stochastic matrix. Thus, via Theorem 3.4, we deduce that the zero state equilibrium of (27) cannot be ESMS-C.
We show now that the zero state equilibrium of (27) is ESMS. We write E[|Θ(t, s)x|2] =x2E[a2(t−1, ηt−1, ηt−2)a2(t−2, ηt−2, ηt−3)...a2(s, ηs, ηs−1)] =
x2 2 it−1=1
2 it−2=1
...
2 is−1=1
a2(t−1, it−1, it−2)a2(t−2, it−2, it−3)...a2(s, is, is−1)
P{ηt−1 =it−1, ηt−2 =it−2, ..., ηs−1=is−1}=
x2P(ηs−1 = 2)a2(s,2,2)...a2(t−2,2,2)ps−1(2,2)ps(2,2)...pt−3(2,2) [a2(t−1,1,2)pt−2(2,1) +a2(t−1,2,2)pt−2(2,2)]≤
x22s−1+s+...+t−3 1
4s+s+1+...t−22t−2 =x22(t+s−3)(2 t−s) 1 2(t+s−2)(t−s−1) ift≥s+ 2. Finally one obtains that
E[|Θ(t, s)x|2]≤x2 1
√2t2−s2−3t−s+4
(29) ift≥s+ 2.
Fort≥s+ 2 we havet2−4t−s2+ 4 = (t−2)2−s2 ≥0. This means that t2−s2−3t−s+ 4≥t−s. From (29) one obtains that
E[|Θ(t, s)x|2]≤x2 1
√2t−s
(30) for all t≥s+ 2, s≥1. Further we compute
E[|Θ(s+ 1, s)x|2] =x2E[a2(s, ηs, ηs−1)] =x2a2(s,1,2)P{ηs−1 = 2, ηs= 1}+ a2(s,2,2)P{ηs−1 = 2, ηs= 2} ≤x22s−1P{ηs−1 = 2}[ps−1(2,1) +ps−1(2,2)].
Thus we get
E[|Θ(s+ 1, s)x|2]≤x22s−1P{ηs−1 = 2}, (∀)s≥1. (31) Take s≥2 and write
E[|Θ(s+1, s)x|2]≤x22s−1[P{ηs−2 = 1}ps−2(1,2)+P{ηs−2 = 2}ps−2(2,2)]≤x22s 1 4s−1 which leads to
E[|Θ(s+ 1, s)x|2]≤x2, ∀s≥2. (32) Further
E[|Θ(2,1)x|2] =E[a2(1, η1, η0)]x2 =x2[P{η0= 2, η1= 1}+P{η0 = 2, η1 = 2}] =
=x2P{η0= 2} i.e.
E[|Θ(2,1)x|2]≤x2. (33)
Finally
E[|Θ(s, s)x|2] =x2. (34) Combining (30),(32),(33),(34) we conclude that
E[|Θ(t, s)x|2]≤√ 2 1
√2t−sx2
for all t≥s≥1, x∈R and thus one obtains that the zero state equilibrium of (27) is ESMS.
Example 3.6. Consider the system (1) in the particular caseN = 2, A(t, i,1) = O∈Rn×n, A(t, i,2) =itIn, i∈ {1,2}, t≥1. The transition probability matrix is P =
⎛
⎝ 1 0 1 0
⎞
⎠. We have
"
p(2,1)A(t,1,2) =tIn
thus via Corollary 3.1 one obtains that the zero state equilibrium of the con- sidered system cannot be SESMS.
On the other hand we have P{ηt = 2} = 0 a.s. for all t ≥ 1. This leads toηt = 1a.s., t≥1. Hence Θ(t, s) = 0a.s. ift≥max{3, s+ 1}, s≥1 for any Markov chain ({ηt}t≥0, P,{1,2}). This shows that in this particular case the zero state equilibrium of the considered system is both ESMS-C, as well as ESMS.
3.2. THE PERIODIC CASE
In this subsection we show that under the periodicity assumption the con- cepts of exponential stability introduced by Definitions 2.3, 2.4, 2.5 become equivalent.
Firstly we introduce
Definition 3.7. We say that the zero state equilibrium of the system (1) is asymptotically stable in mean square ASMS if for any Markov chain ({ηt}t≥0,{Pt}t≥0,D) we have
tlim→∞E[|Θ(t,1)x|2] = 0,∀x∈Rn.
Now we are in position to state
Theorem 3.8. Assume that there exists an integer θ≥1 such that A(t+ θ, i, j) = A(t, i, j), i, j ∈ D, Pt+θ = Pt, t ≥ 0. Under these conditions the following are equivalent:
(i) the zero state equilibrium of the system (1) is (SESMS), (ii) the zero state equilibrium of the system (1) is (ESMS-C), (iii) the zero state equilibrium of the system (1) is (ESMS), (iv) the zero state equilibrium of the system (1) is (ASMS),
(v)there exists a Markov chain({ηt}t≥0,{Pt}t≥0,D)withP(η0=i)>0, i∈ D such that
tlim→∞E[|Θ(θt,1)x|2] = 0 (35)
for all x∈Rn,
(vi) ρ[R(θ+ 1,1)]<1 where ρ[·]is the spectral radius.
Proof. is similar to the proof of Theorem 3.7 in [11]. It is omitted for shortness.
Definition 3.9. We say that the system (1) is in the time invariant case if A(t, i, j) =A(i, j) for allt≥1,i, j ∈Dand Pt=P for all t≥0.
In this case we have Υt= Υ, for allt≥1. One sees that the system (1) is in the time invariant case if and only if it is periodic with period θ= 1. Hence, the equivalences from the above theorem hold in the time invariant case too.
In this case, the statement (vi) becomes ρ(Υ)<1.
4. LYAPUNOV TYPE CRITERIA
In this section we present several conditions for exponential stability in mean square expressed in terms of solvability of some suitable systems of linear matrix equations or linear matrix inequations. The results of this section are special cases of those stated in a more general framework in [7]. That is why we present them here without proofs.
4.1. THE GENERAL CASE
In view of Theorem 3.2 it follows that the Lyapunov type criteria are neces- sary and sufficient conditions for SESMS but they are only sufficient conditions for ESMS. Direct from the above Definition 2.3 and Theorem 3.4 in [7] applied to Lyapunov type operators Υt we obtain
Theorem 4.1 Under the considered assumptions the following are equiva- lent:
(i) the zero state equilibrium of the system (1) is SESMS, (ii) the system of backward linear equations
Xt(i) = N j=1
pt−1(i, j)AT(t, j, i)Xt+1(j)A(t, j, i) +In (36) t ≥ 1, i ∈ D has a bounded solution Xt = (Xt(1), ..., Xt(N)) with Xt(i) ≥ In, t≥1,
(iii) there exists a bounded sequence {Yt}t≥1∈SNn and scalars α >0, δ >0 such that
N j=1
pt−1(i, j)AT(t, j, i)Yt+1(j)A(t, j, i)−Yt(i)≤ −αIn (37) Yt(i)≥δIn, t≥1, i∈D.
Remark 4.2. Even if the system (36), ((37)respectively) consists of an infinite number of equations (inequations respectively) the criteria derived in
Theorem 4.1 may be useful to obtain sufficient conditions for ESMS in the general time varying case. This can be illustrated by the next simple example.
Example 4.3Consider the system (1) in the special case n= 1
x(t+ 1) =a(t, ηt, ηt−1)x(t) (38) t≥1, wherea(t, i, j)∈R are such that
sup
t≥1
N j=1
pt−1(i, j)a2(t, j, i)<1. (39) Under this condition the zero state equilibrium of (38) is SESMS.
Indeed, if (39) holds, then the corresponding system (37) associated with (38) is fulfilled for Yt(i) = 1, t≥1, i∈D.
4.2. THE PERIODIC CASE
From Theorem 3.5 in [7] one obtains that if the coefficients of (36) are periodic with period θ, the unique bounded and positive solution of (36) is periodic with the same period θ. Also, if the system of inequalities (37) has a bounded and uniform positive solution then it has a periodic solution too.
This allows us to obtain the following specialized version of Theorem 4.1.
Theorem 4.4 Under the assumptions of Theorem 3.8, with θ ≥ 2, the following are equivalent:
(i) the zero state equilibrium of (1) is ESMS, (ii) the system of linear matrix equations
Xt(i) = N j=1
pt−1(i, j)AT(t, j, i)Xt+1(j)A(t, j, i) +In 1≤t≤θ−1
Xθ(i) = N j=1
pθ−1(i, j)AT(θ, j, i)X1(j)A(θ, j, i) +In (40) i∈Dhas a solution Xt= (Xt(1), ..., Xt(N))with Xt(i)>0.
(iii) there exist positive definite matricesYt(i),1≤t≤θ, i∈D, which solve the following system of LMI’s
N j=1
pt−1(i, j)AT(t, j, i)Yt+1(j)A(t, j, i)−Yt(i)<0, 1≤t≤θ−1 N
j=1
pθ−1(i, j)AT(θ, j, i)Y1(j)A(θ, j, i)−Yθ(i)<0 i∈D. (41) It is easy to see that under the conditions of Theorem 3.8 the sequence Υt can be extended in a natural way by periodicity, to the whole set of integersZ.
In this case we may use Theorem 3.7 (ii) and Theorem 3.9 from [7] to obtain
Theorem 4.5. Under the assumptions of Theorem 3.8 with θ ≥ 2, the following are equivalent:
(i) the zero state equilibrium of the system (1) is ESMS, (ii) the system of linear matrix equations
Xt+1(i) = N j=1
pt−1(j, i)A(t, i, j)Xt(j)AT(t, i, j) +In 1≤t≤θ−1
X1(i) = N j=1
pθ−1(j, i)A(θ, i, j)Xθ(j)AT(θ, i, j) +In (42) i∈Dhas a solution Xt= (Xt(1), ..., Xt(N))such that Xt(i)>0, i∈D,
(iii) there exist positive definite matricesYt(i),1≤t≤θ, i∈D, which solve the following system of LMI’s:
N j=1
pt−1(j, i)A(t, i, j)Yt(j)AT(t, i, j)−Yt+1(i)<0 1≤t≤θ−1 N
j=1
pθ−1(j, i)A(θ, i, j)Yθ(j)AT(θ, i, j)−Y1(i)<0 (43) i∈D.
Remark 4.6. The system of linear equations (40), (42) and the system of linear inequations (41), (43) have ˆnscalar equations (inequations respectively), with ˆn scalar unknowns, where ˆn= n(n2+1)N θ.
4.3. THE TIME INVARIANT CASE
Using Theorem 3.5 (iii), Theorem 3.7 (iii) and Theorem 3.9 in [7] one obtains the following Lyapunov type criteria for exponential stability in mean square for the system (1) in the time invariant case.
Corollary 4.7If the system is in the time invariant case, the following are equivalent:
(i) the zero state equilibrium of the system (1) is ESMS, (ii) the system of linear equations
X(i) = N j=1
p(i, j)AT(j, i)X(j)A(j, i) +In (44) i∈D, has a solution X= (X(1), ..., X(N))with X(i)>0, i∈D,
(iii) there exist positive definite matrices Y(i), i ∈ D, which solve the fol- lowing system of LMI’s:
N j=1
p(i, j)AT(j, i)Y(j)A(j, i)−Y(i)<0 (45) i∈D.
Corollary 4.8. Under the conditions of Corollary 4.7 the following are equivalent:
(i) the zero state equilibrium of the system (1) is ESMS, (ii) the system of linear matrix equations
X(i) = N j=1
p(j, i)A(i, j)X(j)AT(i, j) +In (46) i∈Dhas a solution X= (X(1), ..., X(N))with X(i)>0, i∈D,
(iii) there exist positive definite matrices Y(i), i ∈ D, which solve the fol- lowing system of LMI’s
N j=1
p(j, i)A(i, j)Y(j)AT(i, j)−Y(i)<0 (47) i∈D.
Remark 4.9. The system of linear equations (44) and (46) and the sys- tem of linear inequations (45) and (47) have ˇn scalar equations (inequations respectively) with ˇnscalar unknowns, where ˇn= n(n2+1)N.
5. APPLICATIONS
In this section we illustrate the applicability of the results concerning the exponential stability in mean square for the systems of type (1) derived in the previous sections to the control problem of the designing of a stabilizing static output feedback in the presence of some delays in the transmission of the date.
We restrict our attention to the time-invariant case of the controlled system (3) and of the control law (4). More specific we consider the controlled system x(t+ 1) =A(t, ηt)x(t) +B(t, ηt)u(t)y(t) =C(t, ηt)x(t), t≥0, (48) and the control law
u(t) =K(ηt)y(t). (49)
Based on Theorem 3.8 (for θ = 1) one obtains that (13) is equivalent to exponential stability in mean square. Therefore to find some conditions which guarantee the existence of the feedbacks gainsK(i) with the desired property, we may apply the Lyapunov type criteria derived in Section 4.
The main tools in the derivation of the results in this section are the fol- lowing well-known lemmas.
Lemma 5.1 (The projection lemma) [23]. Let Z = ZT ∈ Rn×n,U ∈ Rm×n,V ∈ Rp×n be given matrices, n ≥ max{m, p}. Let U⊥,V⊥ be full
column rank matrices such that UU⊥ = 0 and VV⊥ = 0. Then the following are equivalent:
(i) the linear matrix inequations
Z+UTKV+VTKTU<0 with the unknown matrix K ∈Rm×p is solvable;
(ii) (U⊥)TZU⊥<0;
(V⊥)TZV⊥|<0.
Lemma 5.2. (Finsler’s lemma)[23]. LetZ=ZT ∈Rn×n,C∈Rp×n, n > p be given. Take C⊥ a full column rank matrix such that CC⊥ = 0. Then the following are equivalent:
(i) there exists a scalar μ such thatZ+μCTC<0;
(ii) (C⊥)TZC⊥<0.
Combining the above two lemmas one obtains
Corollary 5.3. With the previous notations the following statements are equivalent:
(i) the linear inequation
Z+UTKV+VTKTU<0 (50) with the unknown K∈Rm×p is solvable.
(ii) there exist the scalars μ1, μ2 such that Z+μ1UTU<0 Z+μ2VTV<0.
In [23] a parametrization of the whole class of solutions of (50) is given.
Before to state the main results of this section we remark that in (14) written for the special case of (48) and (49) we have the following decomposition
A(i, j) =˜ A0(i) +B0(i)K(i)C0(j),A(i, j) =ˆ A0(i) +B0(i)K(j)C0(j), (51)
whereA(i) =
⎛
⎝ A0(i) 0 In 0
⎞
⎠∈R2n×2n,B0(i) =
⎛
⎝ B(i) 0
⎞
⎠∈R2n×m,C0(j) = (0 C(j))∈Rp×2n.
Now we have
Theorem 5.4. Assume that there exist the symmetric matrices Y(i) ∈ R2n×2n and the scalarsμ1(i) andμ2(i), i∈D, satisfying the following systems
of LMI’s ⎛
⎜⎜
⎜⎝
−Y(i) Ψ2i(Y) 0 ΨT2i(Y) Ψ3i(Y) Ψ4i(Y)
0 ΨT4i(Y) −μ1(i)Ip
⎞
⎟⎟
⎟⎠<0, (52)
N j=1
p(j, i)A0(i)Y(j)AT0(i)−Y(i) +μ2(i)B0(i)BT0(i)<0, (53) where
Ψ2i(Y) = ("
p(1, i)A0(i)Y(1), ...,"
p(N, i)A0(i)Y(N)), Ψ3i(Y) =−diag(Y(1), ..., Y(N)),
ΨT4i(Y) = ("
p(1, i)C0(1)Y(1), ...,"
p(N, i)C0(N)Y(N)).
Under these conditions there exist stabilizing feedback gains K(i)∈Rm×p, i∈ D such that the zero state equilibrium of the time-invariant version of the system (10) is ESMS.
Moreover the matrices K(i) may be obtained as solutions of the following uncoupled LMI’s
Z˜(i) + ˜UT(i)K(i)˜V(i) + ˜VT(i)KT(i)U(i)<0, (54) i∈D, where
Z˜(i) =
⎛
⎝ −Y˜(i) Ψ2i( ˜Y) ΨT2i( ˜Y) Ψ3i( ˜Y)
⎞
⎠, U˜(i) = (BT0(i),0, ...,0)∈Rmטn,
V˜(i) = (0,"
p(1, i)C0(1) ˜Y(1), ...,"
p(N, i)C0(N) ˜Y(N))∈Rpטn, (55) where n˜ = 2n(N + 1),Y˜ = ( ˜Y(1), ...,Y˜(N)) being a solution of (52), (53).
Proofis similar to the one of Theorem 6.4 in [11]. It is omitted for shortness.
Let us consider the time-invariant case of control law (7).
In this case we have
Theorem 5.5. Assume that there exist matrices Y(i) ∈ R2n×2n and the scalars μ1(i), μ2(i) which satisfy the following system of LMI’s
N j=1
p(i, j)AT0(j)Y(j)A0(j)−Y(i) +μ1(i)CT0(i)C0(i)<0, (56)
⎛
⎜⎜
⎜⎝
−Y(i) Γˆ2i(Y) 0 (ˆΓ2i)T(Y) Γ3i(Y) Γ4i(Y)
0 ΓT41(Y) −μ2(i)Im
⎞
⎟⎟
⎟⎠<0, (57)
where
Γˆ2i(Y) = ("
p(i,1)AT0(1)Y(1), ...,"
p(i, N)AT0(N)Y(N)), Γ3i(Y) = Ψ3i(Y),
ΓT4i(Y) = ("
p(i,1)BT0(1)Y(1), ...,"
p(i, N)B0(N)Y(N)).
Under these conditions there exist feedback gains K(i)∈Rm×p such that the zero state equilibrium of the corresponding time invariant version of the system (11) is ESMS.
Moreover, if ( ˆY(i),μˆ1(i),μˆ2(i), i ∈ D) is a solution of (56), (57) then for each i, K(i) is obtained as a solution of the following LMI
Zˆ(i) + ˆUT(i)K(i)ˆV(i) + ˆVT(i)KT(i) ˆU(i)<0, (58) where
Zˆ(i) =
⎛
⎝ −Yˆ(i) Γˆ2i( ˆY) ΓˆT2i( ˆY) Γ3i( ˆY)
⎞
⎠,Uˆ(i) = (C0(i),0, ...,0)∈Rpטn
Vˆ(i) = (0,"
p(i,1)BT0(1) ˆY(1), ...,"
p(i, N)BT0(N) ˆY(N))∈Rmטn. (59) Proof: see the proof of Theorem 6.4 in [11].
Remark 5.6. Remark that in order to obtain uncoupled LMIs (54) and (58) respectively, we used Corollary 4.8, in the first case and Corollary 4.7 in the second case. We feel that this is a good motivation to deduce stability criteria based on Lyapunov type operators Υt (see Theorem 4.5 and Corollary 4.8) as well as based on its adjoint operators Υ∗t (see Theorem 4.4 and Corollary 4.7).
Acknowledgements
This paper was supported by the Grant no. 2-CEx06-11-18 / 2006, of the Romanian Ministry of Education and Research.
This work was partially supported by Institute of Mathematics of the Ro- manian Academy, Bucharest.
References
[1] A. El Bouhtouri, D. Hinrichsen, A. J. Pritchard, H∞ type control for discrete-time stochastic systems,Int. J. Robust Nonlinear Control,9(1999), 923-948.
[2] O. L. V. Costa, M. D. Fragoso,Necessary and sufficient conditions for mean square stability of discrete-time linear systems subject to Markovian jumps, Proc. 9-th Int.
Symp. on Math. Theory of Networks and Systems, (1991), Kobe, Japan, 85-86.
[3] O. L. V. Costa, M. D. Fragoso,Stability results for discrete-time linear systems with Markovian jumping parameters,J. Math. Anal. Appl.,179, 2 (1993), 154-178.
[4] O. L. V. Costa, R. P. Marques, Mixed H2/H∞ Control of discrete time Markovian jump linear systems,IEEE Trans. Automatic Control,AC-43, 1 (1998), 95-100.
[5] O. L. V. Costa, M. D. Fragoso, R. P. Marques, Discrete-time Markov jump linear systems,, Springer, New York, 2005.
[6] J. L. Doob,Stochastic processes, Wiley, New-York, 1967.
[7] V. Dragan, T. Morozan,Exponential stability for discrete time linear equations defined by positive operators,Integral Equ. Oper. Theory,54, (2006), 465-493.
[8] V. Dragan, T. Morozan,Mean square exponential stability for some stochastic linear discrete time systems,European Journal of Control,12, 4 (2006), 373-395.
[9] V. Dragan, T. Morozan, A. M. Stoica,Mathematical methods in robust control of linear stochastic systems, Mathematical Concepts and Methods in Science and Engineering Series, Springer, New York, 2006.
[10] V. Dragan, T. Morozan,Mean square exponetial stability for discrete-time time-varying linear systems with markovian switching,Proceedings of 17th International Symposium on Mathematical Theory of Networks and Systems, Kyoto, Japan, 2006, CD-Rom.
[11] V. Dragan, T. Morozan, Exponential stability in mean square for a general class of discrete-time linear stochastic systems, Preprint nr. 2/2007, Institute of Mathematics
”Simion Stoilow” of the Romanian Academy.
[12] Y. Fang, K. Loparo, Stochastic stability of jump linear systems, IEEE Trans. Aut.
Control, 47,7(2002), 1204-1208.
[13] X. Feng, K. Loparo, Stability of linear Markovian jump systems, Proc. 29-th IEEE Conf. Decision Control, 1408, Honolulu, Hawaii, 1990.
[14] X. Feng, K. Loparo, Y. Ji, H. J. Chizeck,Stochastic stability properties of jump linear systems,IEEE Trans. Aut. Control,37, 1 (1992), 38-53.
[15] Y. Ji, H. J. Chizeck, X. Feng, K. Loparo, Stability and control of discrete-time jump linear systems,Control Theory and Advances Tech.,7, 2 (1991), 247-270.
[16] R. Z. Khasminskii,Stochastic stability of differential equations; Sythoff and Noordhoff:
Alpen aan den Rijn, NL, 1980.
[17] R. Krtolica, U. Ozguner, H. Chan, H. Goktas, J. Winkelman, M. Liubakka,Stability of linear feedback systems with random communication delays,Proc. 1991 ACC, Boston, MA., June 26-28, 1991.
[18] M. Mariton, Jump linear systems in Automatic control, Marcel Dekker, New-York, 1990.
[19] T. Morozan, Stability and control of some linear discrete-time systems with jump Markov disturbances,Rev. Roum. Math. Rures et Appl.,26, 1 (1981), 101-119.
[20] T. Morozan, Optimal stationary control for dynamic systems with Markovian pertur- bations,Stochastic Anal. and Appl.,1, 3 (1983), 299-325.
[21] T. Morozan,Stability and control for linear discrete-time systems with Markov pertur- bations,Rev. Roum. Math. Pures et Appl.,40, 5-6 (1995), 471-494.
[22] T. Morozan,Dual linear quadratic problem for linear control systems with jump Markov perturbations,Rev. Roum.Math. Pures et Appl.,41, 5-6 (1996), 363-377.
[23] R. E. Skelton, T. Iwasaki, K. Grigoriadis,A unified algebraic approach to linear control design, Taylor and Francis, 1998.
[24] J. Yong, X. Y. Zhou, Stochastic controls. Hamiltonian systems and HJB equations, Springer, New York, 1999.
