Skip to main content
main-content

Über dieses Buch

This book has been written for several reasons, not all of which are academic. This material was for many years the first half of a book in progress on information and ergodic theory. The intent was and is to provide a reasonably self-contained advanced treatment of measure theory, prob ability theory, and the theory of discrete time random processes with an emphasis on general alphabets and on ergodic and stationary properties of random processes that might be neither ergodic nor stationary. The intended audience was mathematically inc1ined engineering graduate students and visiting scholars who had not had formal courses in measure theoretic probability . Much of the material is familiar stuff for mathematicians, but many of the topics and results have not previously appeared in books. The original project grew too large and the first part contained much that would likely bore mathematicians and dis courage them from the second part. Hence I finally followed the suggestion to separate the material and split the project in two. The original justification for the present manuscript was the pragmatic one that it would be a shame to waste all the effort thus far expended. A more idealistic motivation was that the presentation bad merit as filling a unique, albeit smaIl, hole in the literature.

Inhaltsverzeichnis

Frontmatter

l. Probability and Random Processes

Abstract
In this chapter we develop basic mathematical models of discrete time random processes. Such processes are also called discrete time stochastic processes, information sources, and time series. Physically a random process is something that produces a succession of symbols called “outputs” in a random or nondeterministic manner. The symbols produced may be real numbers such as produced by voltage measurements from a transducer, binary numbers as in computer data, two-dimensional intensity fields as in a sequence of images, continuous or discontinuous waveforms, and so on. The space containing all of the possible output symbols is called the alphabet of the random process, and a random process is essentially an assignment of a probability measure to events consisting of sets of sequences of symbols from the alphabet. It is useful, however, to treat the notion of time explicitly as a transformation of sequences produced by the random process. Thus in addition to the common random process model we shall also consider modeling random processes by dynamical systems as considered in ergodic theory.
Robert M. Gray

2. Standard Alphabets

Abstract
It is desirable to develop a theory under the most general possible assumptions. Random process models with very general alphabets are useful because they include all conceivable cases of practical importance. On the other hand, considering only the abstract spaces of the previous chapter can result in both weaker properties and more complicated proofs. Restricting the alphabets to possess some structure is necessary for some results and convenient for others. Ideally, however, we can focus on a class of alphabets that both possesses useful structure and still is sufficiently general to well model all examples likely to be encountered in the real world. Standard spaces are a candidate for this goal and are the topic of this chapter and the next. In this chapter we focus on the definitions and properties of standard spaces, leaving the more complicated demonstration that specific spaces are standard to the next chapter. The reader in a hurry can skip the next chapter. The theory of standard spaces is usually somewhat hidden in theories of topological measure spaces. Standard spaces are related to or include as special cases standard Borel spaces, analytic spaces, Lusin spaces, Suslin spaces, and Radon spaces. Such spaces are usually defined by their relation via a mapping to a complete separable metric space, a topic to be introduced in Chapter 3.
Robert M. Gray

3. Borel Spaces and Polish Alphabets

Abstract
We have seen that standard measurable spaces are the only measurable spaces for which all finitely additive candidate probability measures are also countably additive, and we have developed several properties and some important simple examples. In particular, sequence spaces drawn from countable alphabets and certain subspaces thereof are standard. In this chapter we develop the most important (and, in a sense, the most general) class of standard spaces-Borel spaces formed from complete separable metric spaces. We will accomplish this by showing that such spaces are isomorphic to a standard subspace of a countable alphabet sequence space and hence are themselves standard. The proof will involve a form of coding or quantization.
Robert M. Gray

4. Averages

Abstract
The basic focus of classical ergodic theory was the development of conditions under which sample or time averages consisting of arithmetic means of a sequence of measurements on a random process converged to a probabilistic or ensemble average of the measurement as expressed by an integral of the measurement with respect to a probability measure. Theorems relating these two kinds of averages are called ergodic theorems.
Robert M. Gray

5. Conditional Probability and Expectation

Abstract
We begin the chapter by exploring some relations between measurements, that is, measurable functions, and events, that is, members of a σ-field. In particular, we explore the relation between knowledge of the value of a particular measurement or class of measurements and knowledge of an outcome of a particular event or class of events. Mathematically these are relations between classes of functions and σ-fields. Such relations will be useful in developing properties of certain special functions such as limiting sample averages arising in the study of ergodic properties of information sources. In addition, they are fundamental to the development and interpretation of conditional probability and conditional expectation, that is, probabilities and expectations when we are given partial knowledge about the outcome of an experiment.
Robert M. Gray

6. Ergodic Properties

Abstract
In this chapter we formally define ergodic properties as the existence of limiting sample averages, and we study the implications of such properties. We shall see that if sample averages converge for a sufficiently large class of measurements, e.g., the indicator functions of all events, then the random process must have a property called asymptotic mean stationarity and that there is a stationary measure, called the stationary mean of the process, that has the same sample averages. In addition, it will be seen that the limiting sample averages can be interpreted as conditional probabilities or conditional expectations and that under certain conditions convergence of sample averages implies convergence of the corresponding expectations to a single expectation with respect to the stationary mean. Finally we shall define ergodicity of a process and show that it is a necessary condition for limiting sample averages to be constants instead of random variables.
Robert M. Gray

7. Ergodic Theorems

Abstract
At the heart of ergodic theory are the ergodic theorems: results providing sufficient conditions for dynamical systems or random processes to possess ergodic properties, that is, for sample averages of the form
$$ < f > _n = \tfrac{1}{n}\sum\limits_{i = 0}^{n - 1} {fT^i } $$
to converge to an invariant limit. Traditional developments of the pointwise ergodic theorem focus on stationary systems and use a subsidiary result known as the maximal ergodic lemma (or theorem) to prove the ergodic theorem. The general result for AMS systems then follows since an AMS source inherits ergodic properties from its stationary mean; that is, since the set {x: < f > n (x) converges } is invariant and since a system and its stationary mean place equal probability on all invariant sets, one will possess almost everywhere ergodic properties with respect to a class of measurements if and only if the other one does and the limiting sample averages will be the same.
Robert M. Gray

8. Process Metrics and the Ergodic Decomposition

Abstract
Given two probability measures, say p and m, on a common probability space, how different or distant from each other are they? Similarly, given two random processes with distributions p and m, how distant are the processes from each other and what impact does such a distance have on their respective ergodic properties? The goal of this final chapter is to develop two quite distinct notions of the distance d(p,m) between measures or processes and to use these ideas to delve further into the ergodic properties of processes and the ergodic decomposition. One metric, the distributional distance, measures how well the probabilities of certain important events match up for the two probability measures, and hence this metric need not have any relation to any underlying metric on the original sample space. In other words, the metric makes sense even when we are not putting probability measures on metric spaces. The second metric, the ρ̄-distance (rho-bar distance) depends very strongly on a metric on the output space of the process and measures distance not by how different probabilities are, but by how well one process can be made to approximate another. The second metric is primarily useful in applications in information theory and statistics.
Robert M. Gray

Backmatter

Weitere Informationen