Introduction to Statistical Inference

A typical problem in probability theory is of the following form: A sample space and underlying probability function are specified, and we are asked to compute the probability of a given chance event. For example, if X₁, …, X _n are independent Bernoulli random variables with P{X _i = 1} = 1 − P{X _i = 0} = p, we compute that the probability of the chance event $$\left\{ {\sum\limits_{i = 1}^n {{X_i} = r} } \right\}$$, where r is an integer with $$0 \le r \le n$$, is $$\left( {\begin{array}{*{20}{c}}n\\r\end{array}} \right){p^r}{\left( {1 - p} \right)^{n - r}}$$. In a typical problem of statistics it is not a single underlying probability law which is specified, but rather a class of laws, any of which may possibly be the one which actually governs the chance device or experiment whose outcome we shall observe. We know that the underlying probability law is a member of this class, but we do not know which one it is. The object might then be to determine a “good” way of guessing, on the basis of the outcome of the experiment, which of the possible underlying probability laws is the one which actually governs the experiment whose outcome we are to observe.