Introductory Statistics and Random Phenomena

The principal question addressed in this chapter is how to present in a readable fashion (often) large sets of data, extracting their essential features in a compact and digestible form which, for instance, would permit an easy comparison of different data sets, discern trends, facilitate management and engineering decisions, or predict future behavior. This is what we call the problem of data representation and compression.

In Mathematica Experiment 2.5.1 we observed that the cumulative d.f. and histograms of large samples drawn with finer and finer resolution or, in other words, with smaller and smaller bin size, often seem to smooth out and assume a form that is almost begging to be compressed into a single analytic formula. These various idealized limit relative frequency d.f.s, called probability density functions, and the related cumulative probability distribution functions, will be studied in this chapter. We will also learn how to simulate data sets with an a priori given probability density function.

In this chapter we will try to get to the heart of the notion of randomness by showing its fundamental connection with several concepts of algorithmic and computational complexity. Although the discussion illuminates the philosophical underpinnings of the concept of randomness for a concrete string of data, the conclusions are sobering: perfectly random strings cannot be produced by any finite algorithms (read, computers). A practical way out of this dilemma is suggested.

Independently repeated experiments with random outcomes have a formal counterpart in the mathematical concept of statistical independence. The cleanest way to introduce the latter can be found within the framework of Kolmogorov’s axiomatic probability theory which, since the 1930s, became the standard, and by far most widespread, mathematical model of randomness.

The two preceding chapters analyzed the phenomenon of randomness from the viewpoint of algorithmic and computational complexity of a fixed string of data, and in the context of the formal mathematical probability theory based on Kolmogorov’s concept of a sequence of statistically independent random variables. We complete this picture in the present chapter by demonstrating that certain, seemingly deterministic, dynamical systems also exhibit some attributes of randomness such as stability of frequencies and fluctuations. The essential features here are nonlinearity and/or sensitive dependence on initial conditions.

The exploration of experimental data and the reliability of the statistical inference based on these data depend heavily on the selection of the mathematical model and on the design of the data collection method.

Analysis of Variance (ANOVA) is used to test whether the variability in response data taken for different levels of manipulated variables can be attributed just to random fluctuations, or is caused by the impact of different input levels. Such an approach has been briefly discussed in Section 8.5. A more general case, with several manipulated categorical variables (factors), will be sketched in this chapter. It is one of the basic tools in the design and analysis of experiments.