Anzeige
Erschienen in:
2022 | OriginalPaper | Buchkapitel
2. Countable Markov Chains
verfasst von : Michel Benaïm, Tobias Hurth
Erschienen in: Markov Chains on Metric Spaces
Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.
Wählen Sie Textabschnitte aus um mit Künstlicher Intelligenz passenden Patente zu finden. powered by
Markieren Sie Textabschnitte, um KI-gestützt weitere passende Inhalte zu finden. powered by
Abstract
This chapter is a self-contained mini course on Markov chains with a countable state space. The notions of recurrence and transience are introduced and it is shown how these properties can be verified with the help of Lyapunov functions. We then explain how Lyapunov functions can be used to obtain estimates on the moments of hitting times for a point or a finite set. In addition, we state and prove the ergodic theorem for countable chains, which asserts that the law of the chain converges to a unique stationary distribution provided that the chain is positive recurrent and aperiodic. Towards the end of the chapter, renewal processes are briefly investigated.