Introduction to Statistics

Statistics is the science of collecting, describing, and interpreting data, i.e., the tool box underlying empirical research.

Probability theory is concerned with the outcomes of random experiments. These can be either real world processes or thought experiments. In both cases,

Combinatorial theory investigates possible patterns of orderings of finitely many elements, composed groups (sets) of such orderings, and the number of these orderings and groups.

A random variable is a function that assigns (real) numbers to the results of an experiment. Each possible outcome of the experiment (i.e., value of the corresponding random variable) occurs with a certain probability.

In the following section we present some important probability distributions, which are often used in statistics. These distributions can be described using at most three parameters. In general, the greater the number of parameters describing a distribution, the more flexible the distribution will be to model data.

One of the major tasks of statistics is to obtain information about populations. The set of all elements that are of interest for a statistical analysis is called a population. The population must be defined precisely and comprehensively so that one can immediately determine whether an element belongs to it or not.

Assume a given population with distribution function F(x). In general, the distribution and its characteristics or parameters are not known. Suppose we are interested in say the expectation μ and the variance \(\sigma ^{2}\). (Alternatively, if the data are binary, we may be interested in the population proportion π). As outlined previously, we can learn about the population or equivalently its distribution function F, through (random) sampling. The data may then be used to infer properties of the population, hence the term indirect inference. At the outset, it is important to emphasize that the conclusions drawn may be incorrect, particularly if the sample is small, or not representative of the underlying population. The tools of probability may be used to provide measures of the accuracy or correctness of the estimates or conclusions. We will focus on the estimation of unknown parameters or characteristics. Assume \(\theta\) to be the object of interest, then we differentiate two types of procedures: point estimation and interval estimation.

Statistical tests are tools for the analysis of hypotheses about the characteristics of unknown probability distributions or relationships between random variables. If the probability distribution is specified up to a finite set of parameters, testing for the fully specified probability density amounts to testing whether the parameters take on specific values. As the mathematical specification of a class of probability distributions involves writing down a function that contains parameters whose values aren’t known a priori, tests based on postulated parameters that determine the characteristics of a probability distribution are dubbed “parametric” tests. Statistical estimation procedures can be used to obtain estimates of the specific parameter(s) of interest. Statistical test theory provides a means of quantifying the significance of such estimates. Closely related to the choice of the parameter value(s) is the choice of the class of probability distributions. Such a fully specified distribution has to describe reality as accurately and reliably as possible. In practice, the choice of a functional class such as the Normal (or Gaussian) distribution and estimating and testing parameters is an iterative process. Empirical researchers will have to consider various models (alternative distributions) at the explorative stage of the investigation into the nature of the phenomena of interest. However, very often certain probability models are chosen a priori for their tractability rather than on theoretical grounds. When the postulated class of distribution functions is theory-driven in that it is the result of logical deduction from accepted premises, testing for the significance of parameters forms an important part of the verification of scientific theory. Much of empirical research is, however, data-driven in that there is no a priori distribution function.

In the natural sciences, we can often clearly represent the relationship between two variables by means of a function because it has its origin in physical laws.

The main objective of regression analysis is to describe the expectation and dependence of a quantity Y on quantities X ₁, X ₂, …. A one-directional dependence is assumed. This dependence can be expressed as a general regression function of the following form:

A time series is the vector of realizations of a random variable X over the time.