Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Bücher
Zeitschriften
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
ATZ-Fachkongress

Chassis.tech plus 2024


4 Kongresse in einer Veranstaltung

Treffen Sie in München die Fachleute von Chassis, Lenkung, Bremsen sowie Reifen/Räder.
Erweiterte Suche
Anmelden
nach oben

2004 | Buch | 2. Auflage

Introduction Kapitel lesen Erstes Kapitel lesen

Monte Carlo Statistical Methods

verfasst von: Christian P. Robert, George Casella

Verlag: Springer New York

Buchreihe : Springer Texts in Statistics

Enthalten in: Professional Book Archive

Einloggen, um Zugang zu erhalten
insite
SUCHEN

Über dieses Buch

Monte Carlo statistical methods, particularly those based on Markov chains, are now an essential component of the standard set of techniques used by statisticians. This new edition has been revised towards a coherent and flowing coverage of these simulation techniques, with incorporation of the most recent developments in the field. In particular, the introductory coverage of random variable generation has been totally revised, with many concepts being unified through a fundamental theorem of simulation

There are five completely new chapters that cover Monte Carlo control, reversible jump, slice sampling, sequential Monte Carlo, and perfect sampling. There is a more in-depth coverage of Gibbs sampling, which is now contained in three consecutive chapters. The development of Gibbs sampling starts with slice sampling and its connection with the fundamental theorem of simulation, and builds up to two-stage Gibbs sampling and its theoretical properties. A third chapter covers the multi-stage Gibbs sampler and its variety of applications. Lastly, chapters from the previous edition have been revised towards easier access, with the examples getting more detailed coverage.

This textbook is intended for a second year graduate course, but will also be useful to someone who either wants to apply simulation techniques for the resolution of practical problems or wishes to grasp the fundamental principles behind those methods. The authors do not assume familiarity with Monte Carlo techniques (such as random variable generation), with computer programming, or with any Markov chain theory (the necessary concepts are developed in Chapter 6). A solutions manual, which covers approximately 40% of the problems, is available for instructors who require the book for a course.

Christian P. Robert is Professor of Statistics in the Applied Mathematics Department at Université Paris Dauphine, France. He is also Head of the Statistics Laboratory at the Center for Research in Economics and Statistics (CREST) of the National Institute for Statistics and Economic Studies (INSEE) in Paris, and Adjunct Professor at Ecole Polytechnique. He has written three other books and won the 2004 DeGroot Prize for The Bayesian Choice, Second Edition, Springer 2001. He also edited Discretization and MCMC Convergence Assessment, Springer 1998. He has served as associate editor for the Annals of Statistics, Statistical Science and the Journal of the American Statistical Association. He is a fellow of the Institute of Mathematical Statistics, and a winner of the Young Statistician Award of the Société de Statistique de Paris in 1995.

George Casella is Distinguished Professor and Chair, Department of Statistics, University of Florida. He has served as the Theory and Methods Editor of the Journal of the American Statistical Association and Executive Editor of Statistical Science. He has authored three other textbooks: Statistical Inference, Second Edition, 2001, with Roger L. Berger; Theory of Point Estimation, 1998, with Erich Lehmann; and Variance Components, 1992, with Shayle R. Searle and Charles E. McCulloch. He is a fellow of the Institute of Mathematical Statistics and the American Statistical Association, and an elected fellow of the International Statistical Institute.

Inhaltsverzeichnis

Frontmatter
1. Introduction
Abstract
Until the advent of powerful and accessible computing methods, the experimenter was often confronted with a difficult choice. Either describe an accurate model of a phenomenon, which would usually preclude the computation of explicit answers, or choose a standard model which would allow this computation, but may not be a close representation of a realistic model. This dilemma is present in many branches of statistical applications, for example, in electrical engineering, aeronautics, biology, networks, and astronomy. To use realistic models, the researchers in these disciplines have often developed original approaches for model fitting that are customized for their own problems. (This is particularly true of physicists, the originators of Markov chain Monte Carlo methods.) Traditional methods of analysis, such as the usual numerical analysis techniques, are not well adapted for such settings.
Christian P. Robert, George Casella
2. Random Variable Generation
Abstract
The methods developed in this book mostly rely on the possibility of producing (with a computer) a supposedly endless flow of random variables (usually iid) for well-known distributions. Such a simulation is, in turn, based on the production of uniform random variables. Although we are not directly concerned with the mechanics of producing uniform random variables (see Note 2.6.1), we are concerned with the statistics of producing uniform and other random variables.
Christian P. Robert, George Casella
3. Monte Carlo Integration
Abstract
While Chapter 2 focussed on developing techniques to produce random variables by computer, this chapter introduces the central concept of Monte Carlo methods, that is, taking advantage of the availability of computer generated random variables to approximate univariate and multidimensional integrals. In Section 3.2, we introduce the basic notion of Monte Carlo approximations as a byproduct of the Law of Large Numbers, while Section 3.3 highlights the universality of the approach by stressing the versatility of the representation of an integral as an expectation.
Christian P. Robert, George Casella
4. Controling Monte Carlo Variance
Abstract
In Chapter 3, the Monte Carlo method was introduced (and discussed) as a simulation-based approach to the approximation of complex integrals. There has been a considerable body of work in this area and, while not all of it is completely relevant for this book, in this chapter we discuss the specifics of variance estimation and control. These are fundamental concepts, and we will see connections with similar developments in the realm of MCMC algorithms that are discussed in Chapters 7–12.
Christian P. Robert, George Casella
5. Monte Carlo Optimization
Abstract
This chapter is the equivalent for optimization problems of what Chapter 3 is for integration problems. Here we distinguish between two separate uses of computer generated random variables. The first use, as seen in Section 5.2, is to produce stochastic techniques to reach the maximum (or minimum) of a function, devising random explorations techniques on the surface of this function that avoid being trapped in a local maximum (or minimum) but also that are sufficiently attracted by the global maximum (or minimum). The second use, described in Section 5.3, is closer to Chapter 3 in that it approximates the function to be optimized. The most popular algorithm in this perspective is the EM (Expectation-Maximization) algorithm.
Christian P. Robert, George Casella
6. Markov Chains
Abstract
In this chapter we introduce fundamental notions of Markov chains and state the results that are needed to establish the convergence of various MCMC algorithms and, more generally, to understand the literature on this topic. Thus, this chapter, along with basic notions of probability theory, will provide enough foundation for the understanding of the following chapters. It is, unfortunately, a necessarily brief and, therefore, incomplete introduction to Markov chains, and we refer the reader to Meyn and Tweedie (1993), on which this chapter is based, for a thorough introduction to Markov chains. Other perspectives can be found in Doob (1953), Chung (1960), Feller (1970, 1971), and Billingsley (1995) for general treatments, and Norris (1997), Nummelin (1984), Revuz (1984), and Resnick (1994) for books entirely dedicated to Markov chains. Given the purely utilitarian goal of this chapter, its style and presentation differ from those of other chapters, especially with regard to the plethora of definitions and theorems and to the rarity of examples and proofs. In order to make the book accessible to those who are more interested in the implementation aspects of MCMC algorithms than in their theoretical foundations, we include a preliminary section that contains the essential facts about Markov chains.
Christian P. Robert, George Casella
7. The Metropolis—Hastings Algorithm
Abstract
This chapter is the first of a series on simulation methods based on Markov chains. However, it is a somewhat strange introduction because it contains a description of the most general algorithm of all. The next chapter (Chapter 8) concentrates on the more specific slice sampler, which then introduces the Gibbs sampler (Chapters 9 and 10), which, in turn, is a special case of the Metropolis–Hastings algorithm. (However, the Gibbs sampler is different in both fundamental methodology and historical motivation.)
Christian P. Robert, George Casella
8. The Slice Sampler
Abstract
While many of the MCMC algorithms presented in the previous chapter are both generic and universal, there exists a special class of MCMC algorithms that are more model dependent in that they exploit the local conditional features of the distributions to simulate. Before starting the general description of such algorithms, gathered under the (somewhat inappropriate) name of Gibbs sampling, we provide in this chapter a simpler introduction to these special kind of MCMC algorithms. We reconsider the fundamental theorem of simulation (Theorem 2.15) in light of the possibilities opened by MCMC methodology and construct the corresponding slice sampler.
Christian P. Robert, George Casella
9. The Two-Stage Gibbs Sampler
Abstract
The previous chapter presented the slice sampler, a special case of a Markov chain algorithm that did not need an Accept–Reject step to be valid, seemingly because of the uniformity of the target distribution. The reason why the slice sampler works is, however, unrelated to this uniformity and we will see in this chapter a much more general family of algorithms that function on the same principle. This principle is that of using the true conditional distributions associated with the target distribution to generate from that distribution.
Christian P. Robert, George Casella
10. The Multi-Stage Gibbs Sampler
Abstract
After two chapters of preparation on the slice and two-stage Gibbs samplers, respectively, we are now ready to envision the entire picture for the Gibbs sampler. We describe the general method in Section 10.1, whose theoretical properties are less complete than for the two-stage special case (see Section 10.2): The defining difference between that sampler and the multi-stage version considered here is that the interleaving structure of the two-stage chain does not carry over. Some of the consequences of interleaving are the fact that the individual subchains are also Markov chains, and the Duality Principle and Rao-Blackwellization hold in some generality. None of that is true here, in the multi-stage case. Nevertheless, the multi-stage Gibbs sampler enjoys many optimality properties, and still might be considered the workhorse of the MCMC world. The remainder of this chapter deals with implementation considerations, many in connection with the important role of the Gibbs sampler in Bayesian Statistics.
Christian P. Robert, George Casella
11. Variable Dimension Models and Reversible Jump Algorithms
Abstract
While the previous chapters have presented a general class of MCMC algorithms, there exist settings where they are not general enough. A particular case of such settings is that of variable dimension models. There, the parameter (and simulation) space is not well defined, being a finite or denumerable collection of unrelated subspaces. To have an MCMC algorithm moving within this collection of spaces requires more advanced tools, if only because of the associated measure theoretic subtleties. Section 11.1 motivates the use of variable dimension models in the setup of Bayesian model choice and model comparison, while Section 11.2 presents the general theory of reversible jump algorithms, which were tailored for these models. Section 11.3 examines further algorithms and methods related to this issue.
Christian P. Robert, George Casella
12. Diagnosing Convergence
Abstract
In previous chapters, we have presented the theoretical foundations of MCMC algorithms and showed that, under fairly general conditions, the chains produced by these algorithms are ergodic, or even geometrically ergodic. While such developments are obviously necessary, they are nonetheless insufficient from the point of view of the implementation of MCMC methods. They do not directly result in methods of controlling the chain produced by an algorithm (in the sense of a stopping rule to guarantee that the number of iterations is sufficient). In other words, while necessary as mathematical proofs of the validity of the MCMC algorithms, general convergence results do not tell us when to stop these algorithms and produce our estimates. For instance, the mixture model of Example 10.18 is fairly well behaved from a theoretical point of view, but Figure 10.3 indicates that the number of iterations used is definitely insufficient.
Christian P. Robert, George Casella
13. Perfect Sampling
Abstract
The previous chapters have dealt with methods that are quickly becoming “mainstream”. That is, analyses using Monte Carlo methods in general, and MCMC specifically, are now part of the applied statistician’s tool kit. However, these methods keep evolving, and new algorithms are constantly being developed, with some of these algorithms resulting in procedures that seem radically different from the current standards
Christian P. Robert, George Casella
14. Iterated and Sequential Importance Sampling
Abstract
This chapter gives an introduction to sequential simulation methods, a collection of algorithms that build both on MCMC methods and importance sampling, with importance sampling playing a key role. We will see the relevance of importance sampling and the limitations of standard MCMC methods in many settings, as we try to make the reader aware of important and ongoing developments in this area. In particular, we present an introduction. to Population Monte Carlo (Section 14.4), which extends these notions to a more general case, and subsumes MCMC methods
Christian P. Robert, George Casella
Backmatter
Metadaten
Titel
Monte Carlo Statistical Methods
verfasst von
Christian P. Robert
George Casella
Copyright-Jahr
2004
Verlag
Springer New York
Electronic ISBN
978-1-4757-4145-2
Print ISBN
978-1-4419-1939-7
DOI
https://doi.org/10.1007/978-1-4757-4145-2