Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben

1999 | Buch

Probability Review Kapitel lesen Erstes Kapitel lesen

Markov Chains

Gibbs Fields, Monte Carlo Simulation, and Queues

verfasst von: Pierre Brémaud

Verlag: Springer New York

Buchreihe : Texts in Applied Mathematics

Enthalten in: Professional Book Archive

Einloggen, um Zugang zu erhalten
insite
SUCHEN

Über dieses Buch

In this book, the author begins with the elementary theory of Markov chains and very progressively brings the reader to the more advanced topics. He gives a useful review of probability that makes the book self-contained, and provides an appendix with detailed proofs of all the prerequisites from calculus, algebra, and number theory. A number of carefully chosen problems of varying difficulty are proposed at the close of each chapter, and the mathematics are slowly and carefully developed, in order to make self-study easier. The author treats the classic topics of Markov chain theory, both in discrete time and continuous time, as well as the connected topics such as finite Gibbs fields, nonhomogeneous Markov chains, discrete- time regenerative processes, Monte Carlo simulation, simulated annealing, and queuing theory. The result is an up-to-date textbook on stochastic processes. Students and researchers in operations research and electrical engineering, as well as in physics and biology, will find it very accessible and relevant.

Inhaltsverzeichnis

Frontmatter
Chapter 1. Probability Review
Abstract
The present chapter reviews the notions and results of probability theory directly useful in the sequel. This book is self-contained and the review of probability that follows is sufficient for the purpose of studying Markov chains.
Pierre Brémaud
Chapter 2. Discrete-Time Markov Models
Abstract
Sequences of independent and identically distributed random variables are stochastic processes, but they are not always interesting as stochastic models because they behave more or less in the same way. In order to introduce more variability, one can allow for some dependence on the past, in the manner of deterministic recurrence equations. Discrete-time homogeneous Markov chains possess the required feature, since they can always be represented—at least distributionwise—by a stochastic recurrence equation X n+1 = f(X n , Z n+1),where {Z n } n≥1 is an i.i.d sequence, independent of the initial state X 0.
Pierre Brémaud
Chapter 3. Recurrence and Ergodicity
Abstract
Consider a Markov chain taking its values in E = ℕ. There is a possibility that for any initial state i ∈ ℕ the chain will never visit i after some finite random time. This is often an undesirable feature. For example, if the chain counts the number of customers waiting in line at a service counter (we shall see Markovian models of waiting lines, or queues, at different places in this book), such a behavior implies that the waiting line will eventually go beyond the limits of the waiting facility. In a sense, the corresponding system is unstable.
Pierre Brémaud
Chapter 4. Long Run Behavior
Abstract
Consider an HMC that is irreducible and positive recurrent. In particular, if its initial distribution is the stationary distribution, it keeps the same distribution at all times. The chain is then said to be in the stationary regime, or in equilibrium, or in steady state.
Pierre Brémaud
Chapter 5. Lyapunov Functions and Martingales
Abstract
The present chapter contains a potpourri of topics around potential theory and martingale theory. More exactly, it is a brief introduction to these topics, with the limited purpose of showing the power of martingale theory and the rich interplay between probability and analysis. An important aspect of this chapter concerns the various results complementing the study of recurrence of Chapter 3. In this respect, the single most important result is Foster’s theorem below.
Pierre Brémaud
Chapter 6. Eigenvalues and Nonhomogeneous Markov Chains
Abstract
When the state space is finite, we can rely on the standard results of linear algebra to study the asymptotic behavior of homogeneous Markov chains. Indeed, the asymptotic behavior of the distribution at time n of the chain is entirely described by the asymptotic behavior of the n-step transition matrix P n and the latter depends on the eigenstructure of P. The Perron-Frobenius theorem detailing the eigenstructure of nonnegative matrices is therefore all that is needed, at least in the theory.
Pierre Brémaud
Chapter 7. Gibbs Fields and Monte Carlo Simulation
Abstract
The Markov property of a stochastic sequence {X n } n ≥0 implies that for all n ≥ 1, X n is independent of (X k , k ∉ {n − 1, n, n + 1)) given (X n −1, X n +1).
Pierre Brémaud
Chapter 8. Continuous-Time Markov Models
Abstract
This section introduces random point processes of which the simplest example is the homogeneous Poisson process. A random point process is, roughly speaking, a countable random set of points of the real line. In most applications to engineering and operations research, a point of a point process is the time of occurrence of some event, and this is why points are also called events. For instance, the arrival times of customers at the desk of a post office or jobs at the central processing unit of a computer are point-process events. In biology, an event can be the time of birth of an organism. In physiology, the firing time of a neuron is also an event. In general, point processes on the line appear in stochastic models where the state of a system is changed by the occurrence of some event. In this case one can use the phrase stochastic systems driven by point pmcesses, and if the state of the system is discrete, one sometimes prefers to talk about stochastic discrete event systems. The basic examples are the Poisson process and the continuous-time Markov chain.
Pierre Brémaud
Chapter 9. Poisson Calculus and Queues
Abstract
The phrase Poisson system is an abbreviation of dynamical system driven by Poisson processes. Poisson systems form a special class of the so-called stochastic discrete-event dynamical systems. Their dynamics are expressed in a manner quite analogous in spirit to the recurrence equation of a discrete-time HMC, Theorem 2.1, Chapter 2. However, for Poisson systems, the white noise is a family of independent Poisson processes, and the functional, which associates to the present value of the state process and to the present value of the noise the future value of the state process, does not have —in general— a closed expression but takes the form of an algorithm.
Pierre Brémaud
Backmatter
Metadaten
Titel
Markov Chains
verfasst von
Pierre Brémaud
Copyright-Jahr
1999
Verlag
Springer New York
Electronic ISBN
978-1-4757-3124-8
Print ISBN
978-1-4419-3131-3
DOI
https://doi.org/10.1007/978-1-4757-3124-8