Markov Chains—Passage Phenomena

My treatment of Markov chains in the three chapters which follow is modelled after the material in “Introduction to Stochastic Processes” by Hoel, Port and Stone (Ref. [28]), and is presented here with their kind permission. I have adopted their notation and style, because I feel it is the best way to introduce Markov chains in the spirit of these notes—namely, an approach which combines intuition (of the dynamics) with probabilistic reasoning. The presentation here is compressed and condensed. For a more leisurely account of this material, replete with many examples, problems and related topics, I recommend the Hoel, Port and Stone text. In addition their text discusses the important topics of differentiation and integration of stochastic processes, Brownian motion and stochastic differential equations, which are not contained herein.