Skip to main content
main-content

Über dieses Buch

Continuous time parameter Markov chains have been useful for modeling various random phenomena occurring in queueing theory, genetics, demography, epidemiology, and competing populations. This is the first book about those aspects of the theory of continuous time Markov chains which are useful in applications to such areas. It studies continuous time Markov chains through the transition function and corresponding q-matrix, rather than sample paths. An extensive discussion of birth and death processes, including the Stieltjes moment problem, and the Karlin-McGregor method of solution of the birth and death processes and multidimensional population processes is included, and there is an extensive bibliography. Virtually all of this material is appearing in book form for the first time.

Inhaltsverzeichnis

Frontmatter

1. Transition Functions and Resolvents

Abstract
A stochastic process \(\left\{ {\left. {X\left( t \right),t \in \left[ {0, + \infty } \right.} \right\}} \right.\), defined on a probability space (Ω, F, Pr), with values in a countable set E (to be called the state space of the process), is called a continuous-time parameter Markov chain if for any finite set \( \leqslant {t_1} < {t_2} < ... < {t_n} < {t_{n + 1}}\) of “times,” and corresponding set \({i_1},{i_2}...,{i_{n - 1}},i,j\) of states in E such that \(\Pr \left\{ {X\left( {{t_n}} \right) = i,X\left( {{t_{n - 1}}} \right) = {i_{n - 1}},...X\left( {{t_1}} \right) = {i_1}} \right\} > 0\), we have
$$\Pr \left\{ {X\left( {{t_{n + 1}}} \right) = j\left| {X\left( {{t_n}} \right)} \right. = i,X\left( {{t_{n - 1}}} \right) = {i_{n - 1}},...X\left( {{t_1}} \right) = {i_1}} \right\} = \Pr \left\{ {X\left( {{t_{n + 1}}} \right) = j\left| {X\left( {{t_n}} \right) = i} \right.} \right\}.$$
(1.1)
Equation (1.1) is called the Markov property. If for all s, t such that \( \leqslant s \leqslant t\) and all i,j ε E the conditional probability \(\Pr \left\{ {X\left( t \right) = j\left| {X\left( s \right)} \right. = i} \right\}\) appearing on the right-hand side of (1.1) depends only on ts, and not on s and t individually, we say that the process \(\left\{ {X\left( t \right),t \in \left[ {0, + \left. \infty \right)} \right.} \right\}\) is homogeneous, or has stationary transition probabilities. In this case, then, \(\Pr \left\{ {X\left( t \right) = j\left| {X\left( s \right) = i} \right.} \right\} = \Pr \left\{ {X\left( {x - s} \right) = j\left| {X\left( 0 \right) = i} \right.} \right\}\), and the function
$${P_{ij}}\left( t \right)\mathop = \limits^{def} \Pr \left\{ {X\left( t \right) = j\left| {X\left( 0 \right)} \right. = i} \right\},ij \in E,t \geqslant 0,$$
is called the transition function of the process.
William J. Anderson

2. Existence and Uniqueness of Q-Functions

Abstract
Suppose that \(\left\{ {{X_n},n = 0,1,...} \right\}\) is a discrete time Markov chain with discrete state space E and stationary transition probabilities. Then the reader is well aware that such a stochastic process is uniquely determined by the one-step transition matrix P whose i,jth component is \({p_{ij = }}\Pr \left\{ {{X_{n + 1}} = j\left| {{X_n} = i} \right.} \right\},\), and an initial distribution vector p, whose ith component is \({p_i} = \Pr \left\{ {{X_0} = i} \right\}.\). Every probability involving the random variables of this chain can be determined from the finite-dimensional distributions \(\Pr \left\{ {{X_{{n_1}}} = {i_1},{X_{{n_2}}} = {i_2},...,{X_{{n_k}}} = {i_k}} \right\}\), and the latter can be expressed in the form
$$\sum\limits_i p iPi{i_1}\left( {{n_1}} \right)P{i_1}{i_2}\left( {{n_2} - {n_1}} \right)...{P_{{i_{k - 1}}}}_{{i_k}}\left( {{n_k} - {n_{k - 1}}} \right),$$
, where P ij (n) is the i,jth element of the n-step transition matrix P n , which is the nth power of the one-step transition matrix P. Furthermore, the one-step transition probabilities P ij have very obvious meaning in terms of the process being modeled by the Markov chain, and are easily estimated from observations of the process.
William J. Anderson

3. Examples of Continuous-Time Markov Chains

Abstract
In this case we have a finite state space E which we can take to be \(\left\{ {1,2,3,...,n} \right\}\). Given any q-matrix Q, which need not be conservative, there is a unique Q-function, and we can solve either the backward or forward equations to find it. Let us consider, for example, the backward equations
$$P'\left( t \right) = QP\left( t \right),P\left( 0 \right) = I.$$
(1.1)
Suppose that Q is similar to the n x n matrix J; that is, that
$$Q = MJ{M^{ - 1}}$$
(1.2)
for some n x n invertible matrix M. Then \(T\left( t \right) = {M^{ - 1}}P\left( t \right)\) satisfies
$$Y'\left( t \right) = JY\left( t \right),Y\left( 0 \right) = {M^{ - 1}}$$
(1.3)
The point is J can in general be a simpler matrix than Q, with the result that the equation in (1.3) is straightforward to solve. The extreme case is when Q has n independent eigenvectors and thus J can be taken to be the diagonal matrix whose diagonal components are the eigenvalues of Q, and M is the matrix whose columns are the corresponding eigenvectors.
William J. Anderson

4. More on the Uniqueness Problem

Abstract
In this chapter, we will be looking more closely at questions of nonuniqueness and uniqueness of q-functions. However, it will be more convenient to work with the Laplace transforms of the quantities involved, particularly the resolvent function in place of the transition function, rather than in the time domain as we did in Chapter 2.
William J. Anderson

5. Classification of States and Invariant Measures

Abstract
Let \({P_{ij}}\left( t \right),ij \in E\), be a standard transition function, and let \(\left\{ {X\left( t \right),t \geqslant 0} \right\}\) denote a continuous-time Markov chain with state space E, and having \({P_{ij}}\left( t \right)\) as its transition function.
William J. Anderson

6. Strong and Exponential Ergodicity

Abstract
The question in this chapter is: Given that the transition function P ij (t) is ergodic (i.e., irreducible and recurrent positive),just how fast can we expect convergence of P ij (t) to the ergodic limits π j ? We shall study two special types of ergodicity, the so-called strong ergodicity and exponential ergodicity. Of course, our main interest is always to characterize these properties in terms of the q matrix.
William J. Anderson

7. Reversibility, Monotonicity, and Other Properties

Abstract
A transition function \({P_{ij}}\left( t \right)\) is said to be weakly symmetric if there exists a set \(\left\{ {{m_i},i \in E} \right\}\) of strictly positive numbers such that
$${m_i}{P_{ij}}\left( t \right) = {m_j}{P_{ji}}\left( t \right)$$
(1.1)
for all
$$i,j \in E$$
(1.1)
and
$$t \geqslant 0.$$
. If, in addition, we have Σ iεE m(i) = 1, then \({P_{ij}}\left( t \right)\) is called symmetric. In either case, the set \(\left\{ {{m_i},i \in E} \right\}\) is called the symmetrizing measure.
William J. Anderson

8. Birth and Death Processes

Abstract
By a set of birth-death parameters, we shall mean a sequence \(\left\{ {\left( {{\lambda _n},{\mu _n}} \right);n = 0,1,2,...} \right\}\) of pairs of numbers such that \({\lambda _n} > \) for all \(n \geqslant 0,{\mu _n} > 0\) for all \(n \geqslant 1,\), and \({\mu _0} \geqslant 0\). In this chapter, Q will represent the birth and death q-matrix of (3.2.1) given by
$$Q = \left( {\begin{array}{*{20}{c}} { - ({\lambda _0} + {\mu _0})}&{{\lambda _0}} \\ {{\mu _1}}&{ - ({\lambda _1} + {\mu _1})} \\ 0&{{\mu _2}} \\ 0&0 \\ \vdots & \vdots \end{array}\begin{array}{*{20}{c}} 0&0& \cdots \\ 0&0& \cdots \\ {{\lambda _2}}&0& \cdots \\ { - ({\lambda _3} + {\mu _3})}&{{\lambda _3}}& \cdots \end{array}} \right)$$
(1.1)
, where \(\left\{ {\left( {{\lambda _n},{\mu _n}} \right);n = 0,1,2,...} \right\}\) is a set of birth-death parameters. Note again that Q is conservative if and only if μ0 = 0, and that if μ 0 > 0, we are allowing the process to jump from state 0 directly to an absorbing state which, given the context here, is most conveniently labeled as — 1.
William J. Anderson

9. Population Processes

Abstract
In this section, we investigate processes with state space \({\mathbb{Z}^ + } = \left\{ {0,1,2,...} \right\}\) which are basically birth and death processes, but which also allow downward jumps called catastrophes, of arbitrary size.
William J. Anderson

Backmatter

Weitere Informationen