Annotated Readings in the History of Statistics

Frontmatter

The Introduction of the Concept of Expectation Comments on Pascal (1654)

Abstract

The notion of the expected value of a gamble or of an insurance is as old as those activities themselves, so that in seeking the origin of “expectation” as it is nowadays understood it is important to be clear about what is being sought. Straightforward enumeration of the fundamental probability set suffices to establish the expected value of a throw at dice, for example, and Renaissance gamblers were familiar enough with the notion of a fair game, in which the expectation of each player is the same so that each should stake the same amount. But when more complicated gambles were considered, as in the Problem of Points, no one was quite sure how to compute the expectation. As is often the case in the development of a new concept, the interest lies not so much in the first stirrings which hindsight reveals but in the way its potential came to be realized and its power exploited. In the case of expectation, Pascal clarified the basic notion and used it to solve the Problem of Points.

H. A. David, A. W. F. Edwards

The First Formal Test of Significance

Comments on Arbuthnott (1710)

Abstract

In the years around 1700 the “argument from design” for the existence of God emerged from the mists of the classical past to become, in the hands of John Arbuthnott, a probability calculation involving the rejection of a null hypothesis on the grounds of the small probability of the observed data given that hypothesis. (The clarifying terminology “null hypothesis” was not coined until 1935, by R.A. Fisher.) The evolution of the argument took place among a small group of Fellows of the Royal Society of London, including Richard Bentley, Abraham de Moivre, Isaac Newton, Samuel Clarke, and William Derham, as well as Arbuthnott himself, leading Hacking (1975, quoting Anders Jeffner) to dub it “Royal Society theology.” By 1718 de Moivre was stating its basis clearly in the Preface to the first edition of The Doctrine of Chances:

Further, The same Arguments which explode the notion of Luck may, on the other side, be useful in some Cases to establish a due comparison between Chance and Design: We may imagine Chance and Design to be as it were in Competition with each other, for the production of some sorts of Events, and may calculate what Probability there is, that those Events should be rather owing to one than to the other.

H. A. David, A. W. F. Edwards

Coincidences and the Method of Inclusion and Exclusion

Comments on Montmort (1713), N. Bernoulli (1713), and de Moivre (1718)

Abstract

The most familiar form of the problem of coincidences or matches goes back to Montmort (1708) and may be stated as follows: successive drawings without replacement are made from randomly shuffled tickets numbered 1,2,..., n. What is the probability of at least one match (i.e., ticket i comes up at the ith draw, i = 1,..., n)?

H. A. David, A. W. F. Edwards

On the Game of Thirteen

Abstract

98. The players first draw a card to determine the banker. Let us suppose that this is Peter and that the number of players is as desired. From a complete pack of fifty-two cards, judged adequately shuffled, Peter draws one after the other, calling one as he draws the first card, two as he draws the second, three as he draws the third, and so on, until the thirteenth, calling king. Then, if in this entire sequence of cards he has not drawn any with the rank he has called, he pays what each of the players has staked and yields to the player on his right.

P. R. de Montmort

Letter from Nicholas Bernoulli to Montmort on the Game of Thirteen, reproduced in Montmort (1713, p. 301)

Abstract

Let the cards held by Peter be denoted by the letters a, b, c, d, e, etc. and let their number be n. The number of all possible cases will be = 1.2.3... n; the number of cases with a in first place = 1.2.3... n - 1; the number of cases with b in second but a not in first place = 1.2.3... n - 1 - 1.2.3... n - 2; the number of cases with c in third place, with neither a in first nor b in second place = 1.2.3... n - 1 - 2 × 1.2.3... n - 2 + 1.2.3... n - 3; the number of cases with d in fourth, none of the preceding being in its correct place = 1.2.3... n - 1 - 3 × 1.2.3... n - 2 + 3 × 1.2.3... n - 3 - 1.2.3... n - 4.

H. A. David, A. W. F. Edwards

The Doctrine of Chances

Abstract

Third Game, when it is eftimated before the Play begins, is \(\frac{{aa}}{{\overline {{{(a + b)}^2}} }} \times \frac{{a - b}}{{a + b}}\ C\).

A. de Moivre

The Determination of the Accuracy of Observations

Comments on Gauss (1816)

Abstract

This influential paper has been reviewed by a good many writers, including Hald (1998, pp. 455–459) to whom we refer the reader. However, we must make a few points here.

H. A. David, A. W. F. Edwards

The Determination of the Accuracy of Observations

Abstract

When making the case for the so-called method of least squares, it is assumed that the probability of an error of observation Δ may be expressed by the formula

$$\frac{h}{{\sqrt\pi}}.{e^{ - hh\Delta \Delta }}$$

(A)

where π is the semi-perimeter of the unit circle, e is the base of natural logarithms, and h is also a constant that according to Section 178 of Theoria Motus Corporum Coelestium may be regarded as a measure of the accuracy of the observations. It is not at all necessary to know the value of h in order to apply the method of least squares to determine the most probable values of those quantities [parameters] on which the observations depend. Also, the ratio of the accuracy of the results to the accuracy of the observations does not depend on h. However, knowledge of its value is in itself interesting and instructive, and I will therefore show how we can arrive at such knowledge through the observations themselves.

C. F. Gauss

The Introduction of Asymptotic Relative Efficiency

Comments on Laplace (1818)

Abstract

In this extract from the second supplement of his famous Théorie analytique des probabilités, Laplace makes, in the context of simple linear regression, a large-sample comparison of what we now call L ₁- and L ₂- estimation. He essentially introduces the notion of asymptotic relative efficiency and, incidentally, pioneers the theory of order statistics.

H. A. David, A. W. F. Edwards

On the Probability of Results Deduced by Methods of any Kind from a Large Number of Observations

Abstract

The preceding methods reduce to multiplication of each equation of condition by a factor, and addition of all the products in order to form a final equation. But we can employ other considerations to obtain the desired result. For example, we can choose that equation of condition approaching most closely to the truth. The procedure I have given in Section 40 of Mécanique céleste is of this kind.

P. S. Laplace

The Logistic Growth Curve

Comments on Verhulst (1845)

Abstract

Interest in a better understanding of the growth of populations was undoubtedly raised by Malthus (1798) in his famous Essay on the Principle of Population. Using hardly any mathematics, Malthus claimed that“population, when unchecked, increases in a geometrical ratio and subsistence in an arithmetical ratio.” It was in this spirit that the Belgian mathematician Pierre-François Verhulst (1804–1849), with some encouragement from his compatriot Adolphe Quetelet, studied a model that included a term slowing population growth

H. A. David, A. W. F. Edwards

Mathematical Investigations on the Law of Population Growth

Abstract

Of all the problems that political economy offers for the consideration of philosophers, one of the most interesting is doubtless the knowledge of the law regulating the progression of the population. To resolve this exactly it would be necessary to assess the influence of the numerous causes that impede or favor the multiplication of the human species. And since several of these causes are variable by nature and by their mode of action, the problem considered in all its generality is clearly insoluble.

P.-F. Verhulst

Goodness-of-Fit Statistics

Comments on Abbe (1863)

Abstract

Kendall (1971) gives an excellent concise account of Abbe’s 1863 paper, especially of the ingenious mathematical techniques used. Naturally he can not, in limited space, capture the full flavor of the paper.

H. A. David, A. W. F. Edwards

On the Conformity-to-a-Law of the Distribution of Errors in a Series of Observations

Abstract

In order to specify completely the assumptions underlying the discussion to follow, we will suppose that we are dealing with some process in which the value of a measurable characteristic B varies with another, A (as, for example, the pressure of a body of steam varies with temperature, or the like). Then, on the one hand, for a series of values a ₁, a ₂… a _n of the determining variable A, let the corresponding values b ₁, b ₂… b _n of B be obtained by direct measurement; on the other hand, suppose that some theoretical formula, expressing the dependence as B = F(A), gives calculated values β ₁, β ₂, β _n corresponding to a ₁,… a _n .¹ The question is now: what is the probability that the n differences

$$ \beta _1 - b_1 = x_1 ...\beta _n - b_n = x_n $$

arise entirely from random errors of observation?

E. Abbe

The Distribution of the Sample Variance Under Normality

Comments on Helmert (1876b)

Abstract

The article translated here is actually just one part, with its own title, of a three-part paper that slays several dragons (Helmert, 1876b). Let X ₁,..., X _n be independent N(µ, σ ²) variates. In the omitted portions Helmert derives essentially the variance of the mean deviation and of the mean difference .

H. A. David, A. W. F. Edwards

The Calculation of the Probable Error from the Squares of the Adjusted Direct Observations of Equal Precision and Fechner’s Formula

Abstract

Let λ denote the deviations of the observations from their arithmetic mean, let σ denote the mean error, and ρ the probable error.

F. R. Helmert

The Random Walk and Its Fractal Limiting Form

Comments on Venn (1888)

Abstract

John Venn, whose name is immortalized in the “Venn diagram” of logic and set theory (1880) which replaced the more confusing “Euler diagram” previously used for the same purpose, published The Logic of Chance in 1866. The third edition, of 1888, contained much new material, including a chapter “The conception of randomness,” in which Venn illustrated the “truly random character” of the digits of π by using them to generate a discrete random walk in two dimensions. He then discussed the limiting form that such a random walk would take if the direction were drawn from a uniform distribution and the step length made indefinitely small, describing the result in terms which we would now encapsulate in the word “fractal.”

H. A. David, A. W. F. Edwards

Estimating a Binomial Parameter Using the Likelihood Function

Comments on Thiele (1889)

Abstract

As was the case with probability in the seventeenth century, likelihood had been emerging as a new concept for some time before it was formalized and named (by Fisher, 1921). Early examples of the method of maximum likelihood have been noticed in the work of J.H. Lambert and Daniel Bernoulli in the eighteenth century and discussed by several authors (see Edwards, 1974; Hald, 1998, Chapter 5; and Stigler, 1999, Chapter 16, for introductions to the literature), and the emergence of the method in modern times has been discussed by Aldrich (1997), Edwards (1997b), and Hald (1998, Chapter 28; 1999).

H. A. David, A. W. F. Edwards

Yule’s Paradox (“Simpson’s Paradox”)

Comments on Yule (1903)

Abstract

George Udny Yule may be described as the father of the contingency table by virtue of his long pioneering paper, “On the association of attributes in statistics” published in the Philosophical Transactions of the Royal Society in 1900. At the time Yule was closely associated with Karl Pearson at University College London, himself then deeply involved in developing coefficients of correlation and other measures of association in 2 × 2 tables arising from the grouping of normally distributed variables (Pearson introduced the term “contingency table” in 1904). A dozen years later the differing approaches of the two men led to an acrimonious public controversy (described by Mackenzie, 1981; see also Magnello, 1993, and Aldrich, 1995) which, however, lies outside our present topic, the creation of spurious associations through combining heterogeneous material.

H. A. David, A. W. F. Edwards

Beginnings of Extreme-Value Theory

Comments on Bortkiewicz (1922a) and von Mises (1923)

Abstract

The distribution of the largest or the smallest of n iid variates naturally has a very long history and goes back at least to Nicholas Bernoulli in 1709. Bernoulli reduces a problem of the expected lifetime of the last survivor among n men to finding the expected value of the maximum of n iid uniform variates. Harter (1978) summarizes this and numerous other early papers that touch on the extremes and the range. Gumbel (1958) gives a brief historical account.

H. A. David, A. W. F. Edwards

Range and Standard Deviation

Abstract

We use range (v) to denote the difference between the largest and the smallest value of a given number (n) of observations of some quantity. Let us focus on the true errors of these observations, i.e., their deviations from the true value, specifically from the mathematical expectation of the quantity in question. If we order these true errors according to their absolute value, i.e., without regard to their sign, beginning with the smallest and concluding with the largest, we obtain a series of positive numbers ε ₁, ε ₂ ... ε _n, where ε _i+1 ≥ ε _i . It follows that v = ε _n + ε _n-1 or v = ε _n + ε _n-1 or v = ε _n + ε _n-3, etc., up to v = ε _n + ε ₁ or v = ε _n - ε ₁, according as already the second largest error has a different sign from the largest or only the third largest or only the fourth largest, etc., or only the smallest, or not a single one of the n - 1 errors. For an arbitrary symmetrical error law the probability is then 1/2 that v =ε _n + ε _n-1, 1/4 that v _n = ε _n + ε _n-1, etc., and finally the probability is 1/2^n-1 that v = ε _n + ε ₁ and with equal probability 1/2^n-1 that v = ε _n - ε ₁.

L. von Bortkiewicz

On the Range of a Series of Observations

Abstract

Allow me to add two remarks to the address by Mr. L. von Bortkiewicz on “Range and Standard Deviation” that was published in these Proceedings.(¹) I believe these remarks will usefully supplement the interesting points made by the speaker. One remark includes a simple and general derivation, not restricted to the assumption of a Gaussian law, of the formula for the expected value of the range. In the second, I provide an asymptotic expression for this expectation, also for a larger class of distributions. For the case of the Gaussian law this expression reduces to a final formula that is very simple, useful in practice, and even instructs us how to find the desired value, without any calculation, in the standard tables of the Gaussian integral.

R. von Mises

The Evaluation of Tournament Outcomes

Comments on Zermelo (1929)

Abstract

This paper by the noted German mathematician Ernst Zermelo (1871–1953) was long overlooked and was brought to the attention of the statistical community only in the mid-1960s, by John Moon and Leo Moser, professors of mathematics at the University of Alberta, Canada. Zermelo is concerned with the evaluation of players in chess tournaments, especially for tournaments lacking the balance of Round Robins, where all pairs of players meet equally often. There had long been an obvious method for dealing with Round Robins, namely to rank players according to their number of wins (counting draws as half-wins).

H. A. David, A. W. F. Edwards

The Evaluation of Tournament Results as a Maximization Problem in Probability Theory

Abstract

Chess tournaments are of late always arranged so that each player meets every other player k times, where k is fixed (usually k = 2, with alternate colors) . The ranking of the players is then determined by the number of games won, where an undecided (“drawn” ) game is counted as half a win for each of the partners. This method of calculation treats all games equally, regardless of the order of play, and consequently reduces chance effects to a minimum. Apparently the procedure has worked very well in practice if only a ranking is of interest, but it fails completely for broken-off tournaments (in which the number of games played is not the same for each participant).

E. Zermelo

The Origin of Confidence Limits

Comments on Fisher (1930)

Abstract

In the history of ideas it is frequently possible, with the advantage of hindsight, to discern earlier examples of new concepts. Their later appreciation often relies on the clarification of thought accompanying the introduction of terms which distinguish previously confused concepts. In statistics a notable example is provided by the separation of probability and likelihood.

H. A. David, A. W. F. Edwards

Backmatter