Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Basic Measure Theory Read chapter Read first chapter

2014 | OriginalPaper | Chapter

17. Markov Chains

Author : Achim Klenke

Published in: Probability Theory

Publisher: Springer London

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

In spite of their simplicity, Markov processes with countable state space (and discrete time) are interesting mathematical objects with which a variety of real-world phenomena can be modeled. We give an introduction to the basic concepts (Markov property, transition matrix, recurrence, transience, invariant distribution) and then study certain examples in more detail. For example, we show how to compute numerically very precisely, the expected number of returns to the origin of simple random walk on multidimensional integer lattices.
The connection with discrete potential theory will be investigated later, in Chapter 19. Some readers might prefer to skip the somewhat technical construction of general Markov processes in Section 17.1.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Infinitely Divisible Distributions
next chapter Convergence of Markov Chains
Literature
20.
21.
Brémaud P (1999) Markov chains. Texts in applied mathematics, vol 31. Springer, New York. Gibbs fields, Monte Carlo simulation, and queues MATH
26.
Chung KL, Fuchs WHJ (1951) On the distribution of values of sums of random variables. Memoirs of the American Mathematical Society, vol 6
27.
Chung KL (1960) Markov chains with stationary transition probabilities. Die Grundlehren der mathematischen Wissenschaften, vol 104. Springer, Berlin MATHCrossRef
37.
Dudley RM (2002) Real analysis and probability. Cambridge studies in advanced mathematics, vol 74. Cambridge University Press, Cambridge. Revised reprint of the 1989 original MATHCrossRef
39.
Durrett R (2010) Probability: theory and examples, 4th edn. Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, Cambridge CrossRef
53.
Feller W (1968) An introduction to probability theory and its applications, vol I, 3rd edn. Wiley, New York MATH
59.
Georgii H-O (2012) Stochastics: introduction to probability theory and statistics. de Gruyter Lehrbuch, 2nd edn. Walter de Gruyter, Berlin CrossRef
60.
Gibbs AL, Su FE (2002) On choosing and bounding probability metrics. Int Stat Rev 70(3):419–435 MATHCrossRef
61.
64.
Grimmett GR, Stirzaker DR (2001) Probability and random processes, 3rd edn. Oxford University Press, New York
66.
Häggström O (2002) Finite Markov chains and algorithmic applications. London Mathematical Society student texts, vol 52. Cambridge University Press, Cambridge MATHCrossRef
80.
Joyce GS (2003) Singular behaviour of the lattice Green function for the d-dimensional hypercubic lattice. J Phys A 36(4):911–921 MathSciNetMATHCrossRef
83.
Kallenberg O (2002) Foundations of modern probability, 2nd edn. Probability and its applications. Springer, New York MATH
84.
Kantorovič LV, Rubinšteĭn GŠ (1958) On a space of completely additive functions. Vestn Leningr Univ 13(7):52–59
91.
Kemeny JG, Snell JL (1976) Finite Markov chains. Undergraduate texts in mathematics. Springer, New York. Reprinting of the 1960 original MATH
100.
116.
Meyn SP, Tweedie RL (1993) Markov chains and stochastic stability. Communications and control engineering series. Springer, London MATHCrossRef
120.
Müller A, Stoyan D (2002) Comparison methods for stochastic models and risks. Wiley series in probability and statistics. Wiley, Chichester MATH
123.
Norris JR (1998) Markov chains. Cambridge series in statistical and probabilistic mathematics. Cambridge University Press, Cambridge. Reprint of the 1997 edition MATH
124.
Nummelin E (1984) General irreducible Markov chains and nonnegative operators. Cambridge tracts in mathematics, vol 83. Cambridge University Press, Cambridge CrossRef
134.
143.
Revuz D (1984) Markov chains, 2nd edn. North-Holland mathematical library, vol 11. North-Holland, Amsterdam MATH
152.
Seneta E (2006) Non-negative matrices and Markov chains. Springer series in statistics. Springer, New York. Revised reprint of the second (1981) edition MATH
158.
Spitzer F (1976) Principles of random walks, 2nd edn. Graduate texts in mathematics, vol 34. Springer, New York CrossRef
160.
168.
171.
Wright S (1931) Evolution in Mendelian populations. Genetics 16:97–159
Metadata
Title
Markov Chains
Author
Achim Klenke
Copyright Year
2014
Publisher
Springer London
Book
Probability Theory

Print ISBN: 978-1-4471-5360-3

Electronic ISBN: 978-1-4471-5361-0

Copyright Year: 2014

DOI
https://doi.org/10.1007/978-1-4471-5361-0_17