Probability

The subject matter of probability theory is the mathematical analysis of random events, i.e., of those empirical phenomena which—under certain circumstance—can be described by saying that:

They do not have deterministic regularity (observations of them do not yield the same outcome);

whereas at the same time

They possess some statistical regularity (indicated by the statistical stability of their frequency).

Let us consider an experiment of which all possible results are included in a finite number of outcomes ω ₁,..., ω _N . We do not need to know the nature of these outcomes, only that there are a finite number N of them.

The models introduced in the preceding chapter enabled us to give a probabilistic-statistical description of experiments with a finite number of outcomes. For example, the triple (Ω, A, P) with and \(p\left( \omega \right) = {p^{\sum {{a_i}} }}{q^{n - \sum {{a_i}} }} \) is a model for the experiment in which a coin is tossed n times “independently” with probability p of falling head. In this model the number N(Ω) of outcomes, i.e. the number of points in Ω, is the finite number 2 ⁿ .

Many of the fundamental results in probability theory are formulated as limit theorems. Bernoulli’s law of large numbers was formulated as a limit theorem; so was the De Moivre-Laplace theorem, which can fairly be called the origin of a genuine theory of probability and, in particular, which led the way to numerous investigations that clarified the conditions for the validity of the central limit theorem. Poisson’s theorem on the approximation of the binomial distribution by the “Poisson” distribution in the case of rare events was formulated as a limit theorem. After the example of these propositions, and of results on the rapidity of convergence in the De Moivre-Laplace and Poisson theorems, it became clear that in probability it is necessary to deal with various kinds of convergence of distributions, and to establish the rapidity of convergence connected with the introduction of various “natural” measures of the distance between distributions. In the present chapter we shall discuss some general features of the convergence of probability distributions and of the distance between them. In this section we take up questions in the general theory of weak convergence of probability measures in metric spaces. The De Moivre-Laplace theorem, the progenitor of the central limit theorem, finds a natural place in this theory. From §3, it is clear that the method of characteristic functions is one of the most powerful means for proving limit theorems on the weak convergence of probability distributions in R ⁿ . In §6, we consider questions of metrizability of weak convergence. Then, in §8, we turn our attention to a different kind of convergence of distributions (stronger than weak convergence), namely convergence in variation. Proofs of the simplest results on the rapidity of convergence in the central limit theorem and Poisson’s theorem will be given in §§10 and 11.

The series \(\sum\nolimits_{n = 1}^\infty {\left( {1/n} \right)} \) diverges and the series \(\sum\nolimits_{n = 1}^\infty {{{\left( { - 1} \right)}^n}} \left( {1/n} \right) \) converges. We ask the following question. What can we say about the convergence or divergence of a series \( \sum\nolimits_{n = 1}^\infty {\left( {{\xi _n}/n} \right)} \), where ξ₁, ξ₂,... is a sequence of independent identically distributed Bernoulli random variables with P(ξ₁ = + 1) = P(ξ ₁= - 1) = 1/2? In other words, what can be said about the convergence of a series whose general term is ± 1/n, where the signs are chosen in a random manner, according to the sequence ξ₁, ξ₂,... ?

Let (Ω ℱ P) be a probability space and \( \xi = \left( {{\xi _1},{\xi _2},...} \right) \) a sequence of random variables or, as we say, a random sequence. Let θ _k ξ denote the sequence \(\left( {{\xi _{k + 1}},{\xi _{k + 2}},...} \right) \).

According to the definition given in the preceding chapter, a random sequence ξ = (ξ ₁, ξ ₂,...) is stationary in the strict sense if, for every set B ∈ ℬ(R ^∞) and every n ≥ 1, .

The study of the dependence of random variables arises in various ways in probability theory. In the theory of stationary (wide sense) random sequences, the basic indicator of dependence is the covariance function, and the inferences made in this theory are determined by the properties of that function. In the theory of Markov chains (§12.of Chapter I; Chapter VIII) the basic dependence is supplied by the transition function, which completely determines the development of the random variables involved in Markov dependence.