Homogeneous Markov Processes with a Countable Number of States

We shall consider a system the state of which at time t is ξ(t). Let the number of possible states be finite or countable. As usual, we design each state by a number i = 0, 1, … . We suppose that the process of the transition of the system from one state into another is caused by chance and obeys the laws described in (1.9), (1.10) with transition probabilities (2.1)$$ \begin{array}{*{20}{c}} {{p_{ij}}\left( t \right) = P\left\{ {\xi \left( t \right) = j\left| {\xi \left( 0 \right) = i} \right.} \right\},\quad i,j = 0,1, \ldots }\\ {\left( {\sum\limits_i {{p_{ij}}\left( t \right) = 1} } \right).} \end{array} $$ We shall call ξ(t), t ≥ 0 a homogeneous Markov process. 1