main-content

## Über dieses Buch

Markov processes play an important role in the study of probability theory. Homogeneous denumerable Markov processes are among the main topics in the theory and have a wide range of application in various fields of science and technology (for example, in physics, cybernetics, queuing theory and dynamical programming). This book is a detailed presentation and summary of the research results obtained by the authors in recent years. Most of the results are published for the first time. Two new methods are given: one is the minimal nonnegative solution, the second the limit transition method. With the help of these two methods, the authors solve many important problems in the framework of denumerable Markov processes.

## Inhaltsverzeichnis

### Chapter I. The First Construction Theorem

Abstract
Let (Ω,F,P) be a complete probability space, and E=(l,2,•••). σ(ω) is a random variable taking values in [0,∞). x(t,ω) is a function taking values in E and defined for all ω∈Ω,t∈[0, σ(ω))). For every t≥0, let
$$\tilde{x}\left( {t,\omega } \right) = \left\{ {\begin{array}{*{20}{c}} {x\left( {t,\omega } \right),} & {t < \sigma \left( \omega \right);} \\ {\Delta ,} & {t \leqslant \sigma \left( \omega \right).} \\ \end{array} } \right.$$
Hou Zhenting, Guo Qingfeng

### Chapter II. The Second Construction Theorem

Abstract
Suppose(Ω,F,P)is a complete probability space,
$${X^{(n)}}(\omega ) = \left\{ {{x^{(n)}}(t,\,\,\omega ),t\,\, < \,\,{\sigma ^{(n)}}(\omega )} \right\}$$
is a sequence of Q-processes of order one on it, and
$${g_n}({X^{(n\,\, + \,\,1)}}(\omega )) = {X^{(n)}}(\omega )$$
.
Hou Zhenting, Guo Qingfeng

### Chapter III. General Theory

Abstract
The purpose of this chapter is to give an exposition of the general theory of minimal nonnegative solutions for systems of nonnegative linear equations.
Hou Zhenting, Guo Qingfeng

### Chapter IV. Calculation

Abstract
Lemma 4.1.1 Let A = (aij) be a primitive matrix of order n, and let r be its maximal eigenvalue. If
$$r \geqslant 1,$$
(4.1.1)
then
$$\sum\limits_{k = 0}^\infty {\,a_{ij}^{(k)}} = \, + \,\infty \,(i,j\, = \,1,\,2,\, \cdots ,\,n),$$
(4.1.2)
where $$a_{\,\,ij}^{(k)}$$ is determined uniquely by $${A^k} = \left( {a_{\,\,ij}^{(k)}} \right)$$.
Hou Zhenting, Guo Qingfeng

### Chapter V. Systems of 1-Bounded Equations

Abstract
In the last chapter, we studied a method for calculating the minimal nonnegative solutions of systems of nonnegative linear equations. But most systems of nonnegative linear equations encountered are of a special type (called systems of 1-bounded equations). Their minimal nonnegative solutions possess special properties, and calculation of the solutions can be greatly simplified. This chapter is devoted to the study of the minimal nonnegative solutions of the systems of equations of this type.
Hou Zhenting, Guo Qingfeng

### Chapter VI. General Theory

Abstract
Let (Ω, F, P) be a probability space; ξ(ω) is a random variable taking values in the set {0, 1, 2, ⋯}; x{ω) is a function taking values in the countable set E = {1, 2, ⋯} and defined for all $$\omega \in \Omega ,0 \leqslant n < \xi (\omega ) + 1$$, there n is a nonnegative integer.
Hou Zhenting, Guo Qingfeng

### Chapter VII. Martin Exit Boundary Theory

Abstract
The theory of the Martin exit boundary for a Markov chain was first established by Doob [12] and then generalized by Hunt. Many works appeared thereafter, but in all those works there are always some restrictions imposed on the Markov chain. For example, in [13] all the states of the Markov chain are assumed to be nonrecurrent, and in [14] it is assumed that there exists at least one state which can reach all the other states. So the Martin exit boundary theory for general Markov chains has not been established. Moreover, some subjects involved in the exit boundary theory have not been studied in detail. For example, only definitions of the atomic exit point and the nonatomic exit point are given, but no effective criteria. The aim of the present chapter is:
(i)
to establish the Martin exit boundary theory for general Markov chains;

(ii)
to present the criteria for the excessive functions, potential functions, minimal excessive functions, minimal potential functions, and minimal harmonic functions, atomic exit points and nonatomic exit points, and for the existence of atomic exit space and nonatomic exit space;

(iii)
to present the criteria for Blackwell decompositions of atomic almost closed sets, completely nonatomic almost closed sets and state spaces.

Hou Zhenting, Guo Qingfeng

### Chapter VIII. Martin Entrance Boundary Theory

Abstract
For simplicity, we shall identify the Markov chain X(ω)= {xn(ω),n <ζ(ω)+1} with its transition probability matrix P = (pij; i,j∈E), so P is also called a Markov chain.
Hou Zhenting, Guo Qingfeng

### Chapter IX. Minimal Q-Processes

Abstract
Assume that X(ω) = {x{t, ω), t < σ(ω)} is a homogeneous denumerable Markov process defined on a complete probability space (Ω, F, P), with the denumerable set E = {1, 2, ⋯} as its minimal state space, (p ij (t), t ≥ 0, i,j∈E) as its standard transition probability matrix, and assume its Q matrix satisfies the relation:
$$0 < \,{q_i}\, \equiv \, - \,{q_{ii}}\, < \, + \,\infty ,\,\,\,\sum\limits_{j \in E} {{q_{ij}}\,0\,\,\,(i\, \in \,E).}$$
(9.1.1)
Hou Zhenting, Guo Qingfeng

### Chapter X. Q-Processes of Order One

Abstract
Part I of this book, indicates that the minimal Q -process and Q -processes of order one are the bases for the study of general Q-processes. In the preceding chapter we studied the minimal Q-process, and in this chapter, we shall proceed to investigate Q-processes of order one.
Hou Zhenting, Guo Qingfeng

### Chapter XI. Arbitrary Q-Processes

Abstract
In this chapter, we shall investigate arbitrary Q-processes on the basis of the study of minimal Q-processes and Q-processes of order one.
Hou Zhenting, Guo Qingfeng

### Chapter XII. Criteria for the Uniqueness of Q-Processes

Abstract
Suppose X = {x(t,ω), t < σ (ω)} is a homogeneous denumerable Markov process defined on a complete probability space (Ω, F, P), with phase space E = {1, 2, ⋯}, and transition probabilities p ij (t), i, jE, t ≥ 0, which are real-valued functions satisfying the following conditions:
$${p_{ij}}(t) \geqslant 0,$$
(12.1.1)
$$\sum\limits_{j \in E} {{p_{ij}}(t) \leqslant }\,1,$$
(12.1.2)
$$\sum\limits_{k \in E} {p{}_{ik}(t){p_{kj}}(s)}= {p_{ij}}(t + s),$$
(12.1.3)
$$\mathop {\lim }\limits_{t \to 0} {p_{ij}}(t) = {p_{ij}}(0) = {\delta _{ij}},$$
(12.1.4)
where $${\delta _{ii}} = 1,\,\,{\delta _{ij}} = 0\,\,(i \ne j)$$.
Hou Zhenting, Guo Qingfeng

### Chapter XIII. Construction of Q-Processes

Abstract
First we stipulate that all the Q-processes that occurred in §§13.1—13.3 possess the Property (D) introduced in §1.1, and their Q-matrices satisfy (9.1.1).
Hou Zhenting, Guo Qingfeng

### Chapter XIV. Qualitative Theory

Abstract
As indicated in §12.1, with respect to any given Q-matrix, the following three fundamental problems should be answered: (A) Whether there exists a Q-process with Q as its density matrix. (B) If one does exist then, what are necessary and sufficient conditions for the Q-process to be unique? (C) If we know that such a Q-process is not unique, then how can we construct all such Q-processes? If we distinguish qualitative and quantitative points of view, the above Problems (A) and (B) are qualitative problems, while Problem (C) is quantitative.
Hou Zhenting, Guo Qingfeng

### Backmatter

Weitere Informationen