Abstract
Chapter 1 of Permutation Statistical Methods provides an introduction to the next 10 chapters, presenting and comparing the two models of statistical inference—the population model and the permutation model—and the three main approaches to permutation statistical methods—exact, moment approximation, and resampling approximation. Advantages of permutation statistical methods are elucidated and recursion techniques are described and illustrated.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
The actual quote, from the Father Brown mystery “The Head of Caesar” by G. K Chesterton is “What we all dread most is a maze with no centre” [68, p. 229].
- 2.
The terms “permutation test” and “randomization test” are often used interchangeably.
- 3.
As was customary in scientific societies at the time, these special papers were printed in advance and circulated to the membership of the Society. Then, only a brief summary was made by the author at the meeting and the remaining time was devoted to a discussion of the paper. By tradition, the “proposer of the vote of thanks” advanced what he thought was commendable about the paper, and the seconder put forward what he thought was not so worthy. Subsequently, there was a general discussion by the Fellows of the Society and often a number of prominent statisticians offered comments, suggestions, or criticisms, all of which were subsequently printed along with the published paper in the journal of the Society [50, p. 41].
- 4.
The experiment was obviously inspired by an actual tea-tasting experiment at the Rothamsted Experimental Station some dozen years prior, where Fisher was employed as a statistician from 1919 to 1933. The woman tasting the tea was Dr. B. Muriel Bristol, an algologist at the Station. For descriptions of the tea-tasting experiment at the Rothamsted Experimental Station, see discussions by Agresti , [2, pp. 91–97], Berry , Johnston , and Mielke [41, pp. 58–61, 429–432], Box [48], Box [49, pp. 134–135], Fisher [119, pp. 11–29], Fisher [121, Chap. 6], Gridgeman [155], Hall [165], Lehmann [236, pp. 63–64], Okamoto [324], Salsburg [361, pp. 1–2], Senn [369–371], and Springate [384].
- 5.
For a concise summary of the Zea mays experiment, see an informative discussion by Erich Lehmann in his posthumously published 2011 book on Fisher, Neyman, and the Creation of Classical Statistics [236, pp. 65–66].
- 6.
Olaf Tedin (1898–1966) was a Swedish geneticist who spent most of his professional career as a plant breeder with the Swedish Seed Association, Svalöf, where he was in charge of breeding barley and fodder roots in the Weibullsholm Plant Breeding Station, Landskrona.
- 7.
For a brief history of R.A. Fisher and the origins of α = 0. 05, see a 2011 book by Erich Lehmann on Fisher, Neyman, and the Creation of Classical Statistics [236].
- 8.
It was the Pearson type III distribution that Student (W.S. Gosset) used to fit the distribution of sample variances in his classic 1908 article on “The probable error of a mean” [390, p. 4].
- 9.
The Pearson type III distribution was one of four distributions introduced by Karl Pearson in 1895 [333], although the type III distribution had previously been presented without discussion by Pearson in 1893 [332, p. 331]. The type V distribution introduced by Pearson in 1895 was simply the normal distribution and the Pearson type I distribution was a generalized beta distribution.
- 10.
Mielke , Berry , and Brier were not the first to adopt the Pearson type III distribution to approximate a discrete permutation distribution. For example, B.L. Welch utilized the Pearson type III distribution in a 1936 paper on the specification of rules for rejecting too variable a product [417] and used it again in a 1938 paper on testing the significance of differences between the means of two independent samples when the population variances were unequal [419].
- 11.
For a one-way analysis of variance utilizing a moment-approximation approach, see a 1983 article by Berry and Mielke [23].
- 12.
It is generally accepted that the term “Monte Carlo” method was coined by Stanislaw Ulam , John von Neumann , and Nicholas Metropolis in 1946 while they were working on nuclear weapon projects at the Los Alamos National Laboratory [278, 415]. However, in a 2012 book on Turing’s Cathedral, George Dyson attributes the coining of the term “Monte Carlo” solely to Nicholas Metropolis [102, p. 192].
- 13.
It should be noted that the 1957 Dwass article on modified randomization tests for non-parametric hypotheses relied heavily on the theoretical contributions of an article titled “On the theory of some non-parametric hypotheses” by Erich Lehmann and Charles Stein published in The Annals of Mathematical Statistics in 1949 [237].
- 14.
The Mehta–Patel network algorithm was subsequently applied to many more statistical analyses than the highly limited analysis of r×c contingency tables.
- 15.
For a detailed description of the Mehta–Patel network enumeration algorithm, see Berry , Johnston , and Mielke [41, pp. 288–293].
- 16.
- 17.
Emphasis in the original.
- 18.
See also a short but comprehensive 2010 article on this topic by Tom Siegfried in Science News [377].
- 19.
A recursive process is one in which items are defined in terms of items of similar kind. Using a recursive relation, a class of items can be constructed from one or a few initial values (a base) and a small number of relationships (rules). For example, given the base, F 0 = 0 and \(F_{1} = F_{2} = 1\), the Fibonacci series {0, 1, 1, 2, 3, 5, 8, 13, 21, …} can be constructed by the recursive rule \(F_{n} = F_{n-1} + F_{n-2}\) for n > 2.
- 20.
Exact probability values in this example are given to six places to demonstrate the accuracy of recursion processes with an arbitrary initial value.
- 21.
The letter F for the analysis of variance (variance-ratio) test statistic was introduced in 1934 by George Snedecor at Iowa State University, much to the displeasure of R.A. Fisher [378, p. 15]. Prior to 1934 the test statistic was indicated by z, the letter originally assigned to it by Fisher .
References
Agresti, A.: Measures of nominal-ordinal association. J. Am. Stat. Assoc. 76, 524–529 (1981)
Agresti, A.: Categorical Data Analysis, 2nd edn. Wiley, New York (2002)
Agresti, A., Finley, B.: Statistical Methods for the Social Sciences. Prentice-Hall, Upper Saddle River (1997)
Agresti, A., Liu, I.: Modeling a categorical variable allowing arbitrarily many category choices. Biometrics 55, 936–943 (1999)
Agresti, A., Liu, I.: Strategies for modeling a categorical variable allowing multiple category choies. Sociol. Method Res. 29, 403–434 (2001)
Altman, D.G., Bland, J.M.: Measurement in medicine: the analysis of method comparison studies. Statistician 32, 307–317 (1983)
Anderson, T.W.: An Introduction to Multivariate Statistical Analysis, 2nd edn. Wiley, New York (1984)
Anderson, T.W.: Two of Harold Hotelling’s contributions to multivariate analysis. Tech. Rep. 40, Stanford University, Stanford (1990)
Anderson, D.R., Sweeney, D.J., Williams, T.A.: Introduction to Statistics: Concepts and Applications. West, New York (1994)
Ansari, A.R., Bradley, R.A.: Rank-sum tests for dispersion. Ann. Math. Stat. 31, 1174–1189 (1960)
Anscombe, F.J.: Rejection of outliers. Technometrics 2, 123–147 (1960)
Arabie, P.: Was Euclid an unnecessarily sophisticated psychologist? Psychometrika 56, 567–587 (1991)
Arbuckle, J., Aiken, L.S.: A program for Pitman’s permutation test for differences in location. Behav. Res. Methods Instrum. 7, 381 (1975)
Author: Resampling Stats User’s Guide. Resampling Stats, Arlington (1999)
Author: StatXact for Windows. Cytel Software, Cambridge (2000)
Bailer, A.J.: Testing variance equality with randomization tests. J. Stat. Comput. Simul. 31, 1–8 (1989)
Bakan, D.: The test of significance in psychological research. Psychol. Bull. 66, 423–437 (1966)
Bakeman, R., Robinson, B.F., Quera, V.: Testing sequential association: estimating exact p values using sampled permutations. Psychol. Methods 1, 4–15 (1996)
Bartko, J.J.: On various intraclass correlation reliability coefficients. Psychol. Bull. 83, 762–765 (1976)
Bartko, J.J.: Measurement and reliability: statistical thinking considerations. Schizophr. Bull. 17, 483–489 (1991)
Bartlett, M.S.: A note on tests of significance in multivariate analysis. Proc. Camb. Philos. Soc. 34, 33–40 (1939)
Bernardin, H.J., Beatty, R.W.: Performance Appraisal: Assessing Human Behavior at Work. Kent, Boston (1984)
Berry, K.J., Mielke, P.W.: Moment approximations as an alternative to the F test in analysis of variance. Br. J. Math. Stat. Psychol. 36, 202–206 (1983)
Berry, K.J., Mielke, P.W.: An APL function for Radlow and Alf’s exact chi-square test. Behav. Res. Methods Instrum. Comput. 17, 131–132 (1985)
Berry, K.J., Mielke, P.W.: Goodman and Kruskal’s tau-b statistic: a nonasymptotic test of significance. Sociol. Methods Res. 13, 543–550 (1985)
Berry, K.J., Mielke, P.W.: Subroutines for computing exact chi-square and Fisher’s exact probability tests. Educ. Psychol. Meas. 45, 153–159 (1985)
Berry, K.J., Mielke, P.W.: A generalization of Cohen’s kappa agreement measure to interval measurement and multiple raters. Educ. Psychol. Meas. 48, 921–933 (1988)
Berry, K.J., Mielke, P.W.: A family of multivariate measures of association for nominal independent variables. Educ. Psychol. Meas. 52, 41–55 (1992)
Berry, K.J., Mielke, P.W.: Spearman’s footrule as a measure of agreement. Psychol. Rep. 80, 839–846 (1997)
Berry, K.J., Mielke, P.W.: Extension of Spearman’s footrule to multiple rankings. Psychol. Rep. 82, 376–378 (1998)
Berry, K.J., Mielke, P.W.: Least absolute regression residuals: analyses of block designs. Psychol. Rep. 83, 923–929 (1998)
Berry, K.J., Mielke, P.W.: Least sum of absolute deviations regression: distance, leverage, and influence. Percept. Mot. Skills 86, 1063–1070 (1998)
Berry, K.J., Mielke, P.W.: Least sum of Euclidean regression residuals: estimation of effect size. Psychol. Rep. 91, 955–962 (2002)
Berry, K.J., Mielke, P.W.: Longitudinal analysis of data with multiple binary category choices. Psychol. Rep. 93, 127–131 (2003)
Berry, K.J., Martin, T.W., Olson, K.F.: Testing theoretical hypotheses: a PRE statistic. Soc. Forces 53, 190–196 (1974)
Berry, K.J., Martin, T.W., Olson, K.F.: A note on fourfold point correlation. Educ. Psychol. Meas. 34, 53–56 (1974)
Berry, K.J., Mielke, P.W., Iyer, H.K.: Factorial designs and dummy coding. Percept. Mot. Skills 87, 919–927 (1998)
Berry, K.J., Mielke, P.W., Mielke, H.W.: The Fisher–Pitman permutation test: an attractive alternative to the F test. Psychol. Rep. 90, 495–502 (2002)
Berry, K.J., Johnston, J.E., Mielke, P.W.: Exact and resampling probability values for measures associated with ordered R by C contingency tables. Psychol. Rep. 99, 231–238 (2006)
Berry, K.J., Johnston, J.E., Mielke, P.W.: An alternative measure of effect size for Cochran’s Q test for related proportions. Percept. Mot. Skills 104, 1236–1242 (2007)
Berry, K.J., Johnston, J.E., Mielke, P.W.: A Chronicle of Permutation Statistical Methods: 1920–2000 and Beyond. Springer, Cham (2014)
Bilder, C.R., Loughin, T.M.: On the first-order Rao–Scott correction of the Umesh–Loughin–Scherer statistic. Biometrics 57, 1253–1255 (2001)
Bilder, C.R., Loughin, T.M., Nettleton, D.: Multiple marginal independence-testing for pick any/c variables. Commun. Stat. Simul. Comput. 29, 1285–1316 (2000)
Biondini, M.E., Mielke, P.W., Berry, K.J.: Data-dependent permutation techniques for the analysis of ecological data. Vegetatio 75, 161–168 (1988). [The name of the journal was changed to Plant Ecology in 1997]
Blalock, H.M.: A double standard in measuring degree of association. Am. Sociol. Rev. 28, 988–989 (1963)
Blattberg, R., Sargent, T.: Regression with non-Gaussian stable disturbances. Econometrica 39, 501–510 (1971)
Borgatta, E.F.: My student, the purist: a lament. Soc. Q. 9, 29–34 (1968)
Box, G.E.P.: Science and statistics. J. Am. Stat. Assoc. 71, 791–799 (1976)
Box, J.F.: R. A. Fisher: The Life of a Scientist. Wiley, New York (1978)
Box, G.E.P.: An Accidental Statistician: The Life and Memories of George E. P. Box. Wiley, New York (2013). [Inscribed “With a little help from my friend, Judith L. Allen”]
Bradbury, I.: Analysis of variance versus randomization: a comparison. Br. J. Math. Stat. Psychol. 40, 177–187 (1987)
Bradley, J.V.: Distribution-free Statistical Tests. Prentice-Hall, Englewood Cliffs (1968)
Bradley, J.V.: A common situation conducive to bizarre distribution shapes. Am. Stat. 31, 147–150 (1977)
Brandeau, M.L., Chiu, S.S.: Parametric facility location on a tree network with an L p norm cost function. Transp. Sci. 22, 59–69 (1988)
Brennan, P.F., Hays, B.J.: The kappa statistic for establishing interrater reliability in the secondary analysis of qualitative clinical data. Res. Nurs. Heal. 15, 153–158 (1992)
Brennan, R.L., Prediger, D.J.: Coefficient kappa: some uses, misuses, and alternatives. Educ. Psychol. Meas. 41, 687–699 (1981)
Brillinger, D.R., Jones, L.V., Tukey, J.W.: The role of statistics in weather resources management. Tech. Rep. II, Weather Modification Advisory Board, United States Department of Commerce, Washington, DC (1978)
Bross, I.D.J.: Is there an increased risk? Fed. Proc. 13, 815–819 (1954)
Brown, G.W., Mood, A.M.: On median tests for linear hypotheses. In: Neyman, J. (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, vol. II, pp. 159–166. University of California Press, Berkeley (1951)
Burr, E.J.: The distribution of Kendall’s score S for a pair of tied rankings. Biometrika 47, 151–171 (1960)
Burry-Stock, J.A., Laurie, D.G., Chissom, B.S.: Rater agreement indexes for performance assessment. Educ. Psychol. Meas. 56, 251–262 (1996)
Campbell, M.J., Gardner, M.J.: Calculating confidence intervals for some non-parametric analyses. Br. Med. J. 296, 1454–1456 (1988)
Capraro, R.M., Capraro, M.M.: Treatments of effect sizes and statistical significance tests in textbooks. Educ. Psychol. Meas. 62, 771–782 (2002)
Capraro, R.M., Capraro, M.M.: Exploring the APA fifth edition Publication Manual’s impact of the analytic preferences of journal editorial board members. Educ. Psychol. Meas. 63, 554–565 (2003)
Carroll, R.M., Nordholm, L.A.: Sampling characteristics of Kelley’s ε 2 and Hays’ \(\hat{\omega }^{2}\). Educ. Psychol. Meas. 35, 541–554 (1975)
Carver, R.P.: The case against statistical significance testing. Harv. Educ. Rev. 48, 378–399 (1978)
Carver, R.P.: The case against statistical significance testing, revisited. J. Exp. Educ. 61, 287–292 (1993)
Chesterton, G.K.: The Complete Father Brown Stories: “The Head of Caesar”. Star Books, Vancouver (2003)
Cochran, W.G.: The comparison of percentages in matched samples. Biometrika 37, 256–266 (1950)
Cohen, J.: A coefficient of agreement for nominal scales. Educ. Psychol. Meas. 20, 37–46 (1960)
Cohen, J.: Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychol. Bull. 70, 213–220 (1968)
Cohen, J.: Statistical Power Analysis for the Behavioral Sciences. Academic Press, New York (1969)
Cohen, J.: Things I have learned (so far). Am. Psychol. 45, 1304–1312 (1990)
Cohen, J.: The earth is round (p < . 05). Am. Psychol. 49, 997–1003 (1994)
Cohen, J., Cohen, P.: Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Erlbaum, Hillsdale (1975)
Colwell, D.J., Gillett, J.R.: Spearman versus Kendall. Math. Gaz. 66, 307–309 (1982)
Conover, W.J.: Practical Nonparametric Statistics, 3rd edn. Wiley, New York (1999)
Conti, L.H., Musty, R.E.: The effects of delta-9-tetrahydrocannabinol injections to the nucleus accumbens on the locomotor activity of rats. In: Arurell, S., Dewey, W.L., Willette, R.E. (eds.) The Cannabinoids: Chemical, Pharmacologic, and Therapeutic Aspects, pp. 649–655. Academic Press, New York (1984)
Coombs, C.H.: A Theory of Data. Wiley, New York (1964)
Costner, H.L.: Criteria for measures of association. Am. Sociol. Rev. 30, 341–353 (1965)
Cramér, H.: Mathematical Methods of Statistics. Princeton University Press, Princeton (1946)
Crittenden, K.S., Montgomery, A.C.: A system of paired asymmetric measures of association for use with ordinal dependent variables. Soc. Forces 58, 1178–1194 (1980)
Cureton, E.E.: Rank-biserial correlation. Psychometrika 21, 287–290 (1956)
Cureton, E.E.: Rank-biserial correlation when ties are present. Educ. Psychol. Meas. 28, 77–79 (1968)
Curran-Everett, D.: Explorations in statistics: standard deviations and standard errors. Adv. Physiol. Educ. 32, 203–208 (2008)
Daniel, W.W.: Statistical significance versus practical significance. Sci. Educ. 61, 423–427 (1977)
Daniels, H.E.: Rank correlation and population models (with discussion). J. R. Stat. Soc. Ser. B Methodol. 12, 171–191 (1950)
Daniels, H.E.: Note on Durbin and Stuart’s formula for E(r s ). J. R. Stat. Soc. Ser. B Methodol. 13, 310 (1951)
Darwin, C.R.: The Effects of Cross and Self Fertilization in the Vegetable Kingdom. John Murray, London (1876)
David, F.N.: Review of “Rank Correlation Methods” by M. G. Kendall. Biometrika 37, 190 (1950)
de Mast, J., Akkerhuis, T., Erdmann, T.: The statistical evaluation of categorical measurements: simple scales, but treacherous complexity underneath (2014). [Originally a paper presented at the First Stu Hunter Research Conference in Heemskerk, Netherlands, March, 2013]
Decady, Y.R., Thomas, D.R.: A simple test of association for contingency tables with multiple column responses. Biometrics 56, 893–896 (2000)
Diekhoff, G.: Statistics for the Social and Behavioral Sciences: Univariate, Bivariate, Multivariate. Brown, Dubuque (1992)
Dielman, T.E.: A comparison of forecasts from least absolute and least squares regression. J. Forecast. 5, 189–195 (1986)
Dielman, T.E.: Corrections to a comparison of forecasts from least absolute and least squares regression. J. Forecast. 8, 419–420 (1989)
Dielman, T.E., Pfaffenberger, R.: Least absolute value regression: necessary sample sizes to use normal theory inference procedures. Decis. Sci. 19, 734–743 (1988)
Dielman, T.E., Rose, E.L.: Forecasting in least absolute value regression with autocorrelated errors: a small-sample study. Int. J. Forecast. 10, 539–547 (1994)
Dodd, D.H., Schultz, R.F.: Computational procedures for estimating magnitude of effects for some analysis of variance designs. Psychol. Bull. 79, 391–395 (1973)
Durbin, J., Stuart, A.: Inversions and rank correlation coefficients. J. R. Stat. Soc. Ser. B Methodol. 13, 303–309 (1951)
Dwass, M.: Modified randomization tests for nonparametric hypotheses. Ann. Math. Stat. 28, 181–187 (1957)
Dwyer, J.H.: Analysis of variance and the magnitude of effect: a general approach. Psychol. Bull. 81, 731–737 (1974)
Dyson, G.: Turing’s Cathedral: The Origins of the Digital Universe. Pantheon/Vintage, New York (2012)
Eden, T., Yates, F.: On the validity of Fisher’s z test when applied to an actual example of non-normal data. J. Agric. Sci. 23, 6–17 (1933)
Edgington, E.S.: Randomization tests. J. Psychol. 57, 445–449 (1964)
Edgington, E.S.: Statistical inference and nonrandom samples. Psychol. Bull. 66, 485–487 (1966)
Edgington, E.S.: Approximate randomization tests. J. Psychol. 72, 143–149 (1969)
Edgington, E.S.: Statistical Inference: The Distribution-Free Approach. McGraw-Hill, New York (1969)
Edgington, E.S.: Randomization Tests. Marcel Dekker, New York (1980)
Edgington, E.S., Onghena, P.: Randomization Tests, 4th edn. Chapman & Hall/CRC, Boca Raton (2007)
Edwards, D.: Exact simulation based inference: a survey, with additions. J. Stat. Comput. Simul. 22, 307–326 (1985)
Everitt, B.S.: Moments of the statistics kappa and weighted kappa. Br. J. Math. Stat. Psychol. 21, 97–103 (1968)
Ezekiel, M.J.B.: Methods of Correlation Analysis. Wiley, New York (1930)
Feinstein, A.R.: Clinical biostatistics XXIII: the role of randomization in sampling, testing, allocation, and credulous idolatry (Part 2). Clin. Pharmacol. Ther. 14, 898–915 (1973)
Feinstein, A.R.: Clinical Biostatistics. C.V. Mosby, St. Louis (1977)
Ferguson, G.A.: Statistical Analysis in Psychology and Education, 5th edn. McGraw-Hill, New York (1981)
Festinger, L.: The significance of differences between means without reference to the frequency distribution function. Psychometrika 11, 97–105 (1946)
Fidler, F., Thompson, B.: Computing correct confidence intervals for ANOVA fixed- and random-effects effect sizes. Educ. Psychol. Meas. 61, 575–604 (2001)
Fisher, R.A.: Statistical Methods for Research Workers. Oliver and Boyd, Edinburgh (1925)
Fisher, R.A.: The Design of Experiments. Oliver and Boyd, Edinburgh (1935)
Fisher, R.A.: The logic of inductive inference (with discussion). J. R. Stat. Soc. 98, 39–82 (1935)
Fisher, R.A.: Mathematics of a lady tasting tea. In: Newman, J.R. (ed.) The World of Mathematics, vol. III, section VIII, pp. 1512–1521. Simon & Schuster, New York (1956)
Fisher, R.A.: The Design of Experiments, 7th edn. Hafner, New York (1960)
Fleiss, J.L.: Estimating the magnitude of experimental effects. Psychol. Bull. 72, 273–276 (1969)
Fleiss, J.L., Cohen, J., Everitt, B.S.: Large sample standard errors of kappa and weighted kappa. Psychol. Bull. 72, 323–327 (1969)
Franklin, L.A.: Exact tables of Spearman’s footrule for n = 11(1)18 with estimate of convergence and errors for the normal approximation. Stat. Probab. Lett. 6, 399–406 (1988)
Freeman, L.C.: Elementary Applied Statistics. Wiley, New York (1965)
Frick, R.W.: Interpreting statistical testing: process and propensity, not population and random sampling. Behav. Res. Methods Instrum. Comput. 30, 527–535 (1998)
Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. J. Am. Stat. Assoc. 32, 675–701 (1937)
Friedman, M.: A comparison of alternative tests of significance for the problem of m rankings. Ann. Math. Stat. 11, 86–92 (1940)
Friedman, H.: Magnitude of experimental effect and a table for its rapid estimation. Psychol. Bull. 70, 245–251 (1968)
Gaebelein, J.W., Soderquist, J.A., Powers, W.A.: A note on the variance explained in the mixed analysis of variance model. Psychol. Bull. 83, 1110–1112 (1976)
Gail, M., Mantel, N.: Counting the number of r × c contingency tables with fixed margins. J. Am. Stat. Assoc. 72, 859–862 (1977)
Gardner, M.J., Altman, D.G.: Statistics with Confidence: Confidence Intervals and Statistical Guidelines. British Medical Journal, London (1989)
Geary, R.C.: Some properties of correlation and regression in a limited universe. Metron 7, 83–119 (1927)
Geary, R.C.: Testing for normality. Biometrika 34, 209–242 (1947)
Gebhard, J., Schmitz, N.: Permutation tests: a revival? I. Optimum properties. Stat. Pap. 39, 75–85 (1998)
Glass, G.V.: Note on rank-buserial correlation. Educ. Psychol. Meas. 26, 623–631 (1966)
Glass, G.V.: Primary, secondary, and meta-analysis of research. Educ. Res. 5, 3–8 (1976)
Glass, G.V.: Statistical Methods in Education and Psychology, 2nd edn. Prentice-Hall, Englewood Cliffs (1984)
Glass, G.V., Hakstian, A.R.: Measures of association in comparative experiments: their development and interpretation. Am. Educ. Res. J. 6, 403–414 (1969)
Glass, G.V., Peckham, P.D., Sanders, J.R.: Consequences of failure to meet assumptions underlying the fixed effects analysis of variance and covariance. Rev. Educ. Res. 42, 237–288 (1972)
Glass, G.V., McGraw, B., Smith, M.L.: Meta-Analysis in Social Research: Individual and Neighbourhood Reactions. Sage, Beverly Hills (1981)
Golding, S.L.: Flies in the ointment: methodological problems in the analysis of the percentage of variance due to persons and situations. Psychol. Bull. 82, 278–289 (1975)
Good, I.J.: Further comments concerning the lady tasting tea or beer: P-values and restricted randomization. J. Stat. Comput. Simul. 40, 263–267 (1992)
Good, P.I.: Permutation, Parametric and Bootstrap Tests of Hypotheses. Springer, New York (1994)
Good, P.I.: Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses. Springer, New York (1994)
Good, P.I.: Resampling Methods: A Practical Guide to Data Analysis. Birkhäuser, Boston (1999)
Good, P.I.: Permutation Tests: A Practical Guide to Resampling Methods for Testing Hypotheses, 2nd edn. Springer, New York (2000)
Good, P.I.: Resampling Methods: A Practical Guide to Data Analysis, 2nd edn. Birkhäuser, Boston (2001)
Good, P.I.: Extensions of the concept of exchangeability and their applications. J. Mod. Appl. Stat. Methods 1, 243–247 (2002)
Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications. J. Am. Stat. Assoc. 49, 732–764 (1954)
Goodman, L.A., Kruskal, W.H.: Measures of association for cross classifications, III: approximate sampling theory. J. Am. Stat. Assoc. 58, 310–364 (1963)
Gravetter, F.J., Wallnau, L.B.: Essentials of Statistics for the Behavioral Sciences, 8th edn. Wadsworth, Belmont (2014)
Greenhouse, S.W., Geisser, S.: On methods in the analysis of profile data. Psychometrika 24, 95–112 (1959)
Gridgeman, N.T.: The lady tasting tea, and allied topics. J. Am. Stat. Assoc. 54, 776–783 (1959)
Grier, D.A.: Statistical laboratories and the origins of computing. Chance 12, 14–20 (1999)
Grissom, R.J., Kim, J.J.: Effect Sizes for Research: A Broad Practical Approach. Lawrence Erlbaum, Mahwah (2005)
Grissom, R.J., Kim, J.J.: Effect Sizes for Research: Univariate and Multivariate Applications. Routledge, New York (2012)
Guggenmoos-Holzmann, I.: How reliable are chance-corrected measures of agreement? Stat. Med 12, 2191–2205 (1993)
Guggenmoos-Holzmann, I.: Comment on “Modeling covariate effects in observer agreement studies: the case of nominal scale agreement” by P. Graham. Stat. Med. 14, 2285–2286 (1995)
Guilford, J.P.: Fundamental Statistics in Psychology and Education. McGraw-Hill, New York (1950)
Hald, A.: History of Probability and Statistics and Their Applications Before 1750. Wiley, New York (1990)
Hald, A.: A History of Mathematical Statistics from 1750 to 1930. Wiley, New York (1998)
Haldane, J.B.S., Smith, C.A.B.: A simple exact test for birth-order effect. Ann. Eugen. 14, 117–124 (1948)
Hall, N.S.: R. A. Fisher and his advocacy of randomization. J. Hist. Biol. 40, 295–325 (2007)
Hanley, J.A.: Standard error of the kappa statistic. Psychol. Bull. 102, 315–321 (1987)
Harding, E.F.: An efficient, minimal-storage procedure for calculating the Mann–Whitney U, generalized U and similar distributions. J. R. Stat. Soc.: Ser. C: Appl. Stat. 33, 1–6 (1984)
Hayes, A.F.: Permutation test is not distribution-free: testing H 0: ρ = 0. Psychol. Methods 1, 184–198 (1996)
Hays, W.L.: Statistics. Holt, Rinehart and Winston, New York (1963)
Hedges, L.V.: Estimation of effect size from a series of independent experiments. Psychol. Bull. 92, 490–499 (1982)
Heiser, W.J.: Geometric representation of association between categories. Psychometrika 69, 513–545 (2004)
Hellman, M.: A study of some etiological factors of malocclusion. Dent. Cosmos 56, 1017–1032 (1914)
Hemelrijk, J.: Note on Wilcoxon’s two-sample test when ties are present. Ann. Math. Stat. 23, 133–135 (1952)
Henson, R.K., Smith, A.D.: State of the art in statistical significance and effect size reporting: a review of the APA task force report and current trends. J. Res. Dev. Educ. 33, 285–296 (2000)
Hess, B., Olejnik, S., Huberty, C.J.: The efficacy of two improvement-over-chance effect sizes for two-group univariate comparisons. Educ. Psychol. Meas. 61, 909–936 (2001)
Higgins, J.J.: Introduction to Modern Nonparametric Tests. Brooks/Cole, Pacific Grove (2004)
Hitchcock, D.B.: Yates and contingency tables: 75 years later. Electron. J. Hist. Probab. Stat. 5, 1–14 (2009)
Hodges, J.L., Lehmann, E.L.: Rank methods for combination of independent experiments in analysis of variance. Ann. Math. Stat. 33, 482–497 (1962)
Hodges, J.L., Lehmann, E.L.: Estimates of location based on rank tests. Ann. Math. Stat. 34, 598–611 (1963)
Hope, A.C.A.: A simplified Monte Carlo significance test procedure. J. R. Stat. Soc. Ser. B Methodol. 30, 582–598 (1968)
Hotelling, H.: The generalization of student’s ratio. Ann. Math. Stat. 2, 360–378 (1931)
Hotelling, H.: A generalized T test and measure of multivariate dispersion. In: Neyman, J. (ed.) Proceedings of the Second Berkeley Symposium on Mathematical Statistics and Probability, vol. II, pp. 23–41. University of California Press, Berkeley (1951)
Hotelling, H., Pabst, M.R.: Rank correlation and tests of significance involving no assumption of normality. Ann. Math. Stat. 7, 29–43 (1936)
Howell, D.C.: Statistical Methods for Psychology, 6th edn. Wadsworth, Belmont (2007)
Howell, D.C.: Statistical Methods for Psychology, 8th edn. Wadsworth, Belmont (2013)
Hubbard, R.: Alphabet soup: Blurring the distinctions between p’s and α’s in psychological research. Theor. Psychol. 14, 295–327 (2004)
Hubert, L.J.: A note on Freeman’s measure of association for relating an ordered to an unordered factor. Psychometrika 39, 517–520 (1974)
Hunter, A.A.: On the validity of measures of association: the nominal-nominal two-by-two case. Am. J. Sociol. 79, 99–109 (1973)
Hutchinson, T.P.: Kappa muddles together two sources of disagreement: Tetrachoric correlation is preferable. Res. Nurs. Health 16, 313–315 (1993)
Huynh, H., Feldt, L.S.: Conditions under which mean square ratios in repeated measurements designs have exact F distributions. J. Am. Stat. Assoc. 65, 1582–1589 (1970)
Irwin, J.O.: Tests of significance for differences between percentages based on small numbers. Metron 12, 83–94 (1935)
Isaacson, W.: The Innovators. Simon & Schuster, New York (2014)
Jockel, K.H.: Finite sample properties and asymptotic efficiency of Monte Carlo tests. J. Stat. Comput. Simul. 14, 336–347 (1986)
Johnston, J.E., Berry, K.J., Mielke, P.W.: A measure of effect size for experimental designs with heterogeneous variances. Percept. Mot. Skills 98, 3–18 (2004)
Johnston, J.E., Berry, K.J., Mielke, P.W.: Permutation tests: precision in estimating probability values. Percept. Mot. Skills 105, 915–920 (2007)
Jonckheere, A.R.: A distribution-free k-sample test against ordered alternatives. Biometrika 41, 133–145 (1954)
Kahaner, D., Moler, C., Nash, S.: Numerical Methods and Software. Prentice-Hall, Englewood Cliffs (1988)
Kaufman, E.H., Taylor, G.D., Mielke, P.W., Berry, K.J.: An algorithm and FORTRAN program for multivariate LAD (ℓ 1 of ℓ 2) regression. Computing 68, 275–287 (2002)
Keller-McNulty, S., Higgins, J.J.: Effect of tail weight and outliers and power and type-I error of robust permutation tests for location. Commun. Stat. Simul. Comput. 16, 17–35 (1987)
Kelley, T.L.: An unbiased correlation ratio measure. Proc. Natl. Acad. Sci. 21, 554–559 (1935)
Kempthorne, O.: The Design and Analysis of Experiments. Wiley, New York (1952)
Kempthorne, O.: The randomization theory of experimental inference. J. Am. Stat. Assoc. 50, 946–967 (1955)
Kempthorne, O.: Some aspects of experimental inference. J. Am. Stat. Assoc. 61, 11–34 (1966)
Kempthorne, O.: Why randomize? J. Stat. Plan. Inference 1, 1–25 (1977)
Kendall, M.G.: A new measure of rank correlation. Biometrika 30, 81–93 (1938)
Kendall, M.G.: The treatment of ties in ranking problems. Biometrika 33, 239–251 (1945)
Kendall, M.G.: Rank Correlation Methods. Griffin, London (1948)
Kendall, M.G.: Rank Correlation Methods, 3rd edn. Griffin, London (1962)
Kendall, M.G., Babington Smith, B.: The problem of m rankings. Ann. Math. Stat. 10, 275–287 (1939)
Kendall, M.G., Babington Smith, B.: On the method of paired comparisons. Biometrika 31, 324–345 (1940)
Kendall, M.G., Kendall, S.F.H., Babington Smith, B.: The distribution of Spearman’s coefficient of rank correlation in a universe in which all rankings occur an equal number of times. Biometrika 30, 251–273 (1939)
Kennedy, P.E.: Randomization tests in econometrics. J. Bus. Econ. Stat. 13, 85–94 (1995)
Kenny, D.A.: Statistics for the Social and Behavioral Sciences. Little Brown, Boston (1987)
Keppel, G.: Design and Analysis: A Researcher’s Handbook, 2nd edn. Prentice-Hall, Englewood Cliffs (1982)
Keppel, G., Zedeck, S.: Data Analysis for Research Designs: Analysis of Variance and Multiple Regression/Correlation Approaches. Freeman, New York (1989)
Kim, M.J., Nelson, C.R., Startz, R.: Mean revision in stock prices? a reappraisal of the empirical evidence. Rev. Econ. Stud. 58, 515–528 (1991)
Kingman, J.F.C.: Uses of exchangeability. Ann. Probab. 6, 183–197 (1978). [Abraham Wald memorial lecture delivered in Aug 1977 in Seattle, Washington]
Kirk, R.E.: Experimental Design: Procedures for the Behavioral Sciences. Brooks/Cole, Belmont (1968)
Kirk, R.E.: Practical significance: a concept whose time has come. Educ. Psychol. Meas. 56, 746–759 (1996)
Kirk, R.E.: Effect magnitude: a different focus. J. Stat. Plan. Inference 137, 1634–1646 (2006). [Keynote address delivered at the 2003 International Conference on Statistics, Combinatorics, and Related Areas, held at the University of Southern Maine]
Kraft, C.A., van Eeden, C.: A Nonparametric Introduction to Statistics. Macmillan, New York (1968)
Krause, E.F.: Taxicab Geometry. Addison-Wesley, Menlo Park (1975)
Krippendorff, K.: Bivariate agreement coefficients for reliability of data. In: Borgatta, E.G. (ed.) Sociological Methodology, pp. 139–150. Jossey-Bass, San Francisco (1970)
Kruskal, W.H.: Historical notes on the Wilcoxon unpaired two-sample test. J. Am. Stat. Assoc. 52, 356–360 (1957)
Kruskal, W.H., Wallis, W.A.: Use of ranks in one-criterion variance analysis. J. Am. Stat. Assoc. 47, 583–621 (1952). [Erratum: J. Am. Stat. Assoc. 48, 907–911 (1953)]
Lachin, J.M.: Statistical properties of randomization in clinical trials. Control. Clin. Trials 9, 289–311 (1988)
LaFleur, B.J., Greevy, R.A.: Introduction to permutation and resampling-based hypothesis tests. J. Clin. Child Adolesc. 38, 286–294 (2009)
Lance, C.E.: More statistical and methodological myths and urban legends. Organ. Res. Methods 14, 279–286 (2011)
Lange, J.: Crime as Destiny: A Study of Criminal Twins. Allen & Unwin, London (1931). [Translated by C. Haldane]
Larson, S.C.: The shrinkage of the coefficient of multiple correlation. J. Educ. Psychol. 22, 45–55 (1931)
Larson, R.C., Sadiq, G.: Facility locations with the Manhattan metric in the presence of barriers to travel. Oper. Res. 31, 652–669 (1983)
Lawley, D.N.: A generalization of Fisher’s z test. Biometrika 30, 180–187 (1938)
Lawley, D.N.: Corrections to “A generalization of Fisher’s z test”. Biometrika 30, 467–469 (1939)
Leach, C.: Introduction to Statistics: A Nonparametric Approach for the Social Sciences. Wiley, New York (1979)
Lehmann, E.L.: Parametrics vs. nonparametrics: two alternative methodologies. J. Nonparametr. Stat. 21, 397–405 (2009)
Lehmann, E.L.: Fisher, Neyman, and the Creation of Classical Statistics. Springer, New York (2011)
Lehmann, E.L., Stein, C.M.: On the theory of some non-parametric hypotheses. Ann. Math. Stat. 20, 28–45 (1949)
Levine, J.H.: Joint-space analysis of “pick-any” data: analysis of choices from an unconstrained set of alternatives. Psychometrika 44, 85–92 (1979)
Levine, T.R., Hullett, C.R.: Eta squared, partial eta squared, and misreporting of effect size in communication research. Hum. Commun. Res. 28, 612–625 (2002)
Levine, T.R., Weber, R., Hullett, C.R., Park, H.S., Massi Lindsey, L.L.: A critical assessment of null hypothesis significance testing in quantitative communication research. Hum. Commun. Res. 34, 171–187 (2008)
Levine, T.R., Weber, R., Park, H.S., Hullett, C.R.: A communication researchers’ guide to null hypothesis significance testing and alternatives. Hum. Commun. Res. 34, 188–209 (2008)
Light, R.J.: Measures of response agreement for qualitative data: some generalizations and alternatives. Psychol. Bull. 76, 365–377 (1971)
Light, R.J., Margolin, B.H.: An analysis of variance for categorical data. J. Am. Stat. Assoc. 66, 534–544 (1971)
Linn, R.L., Baker, E.L., Dunbar, S.B.: Complex performance-based assessment: expectations and validation criterion. Educ. Res. 20, 15–21 (1991)
Loether, H.J., McTavish, D.G.: Descriptive and Inferential Statistics: An Introduction, 4th edn. Allyn and Bacon, Boston (1993)
Loughin, T.M., Scherer, P.N.: Testing for association in contingency tables with multiple column responses. Biometrics 54, 630–637 (1998)
Ludbrook, J.: Advantages of permutation (randomization) tests in clinical and experimental pharmacology and physiology. Clin. Exp. Pharmacol. Physiol. 21, 673–686 (1994)
Ludbrook, J.: Issues in biomedical statistics: comparing means by computer-intensive tests. Aust. N. Z. J. Surg. 65, 812–819 (1995)
Ludbrook, J.: The Wilcoxon–Mann–Whitney test condemned. Br. J. Surg. 83, 136–137 (1996)
Ludbrook, J.: Statistical techniques for comparing measures and methods of measurement: a critical review. Clin. Exp. Pharmacol. Physiol. 29, 527–536 (2002)
Ludbrook, J.: Outlying observations and missing values: how should they be handled? Clin. Exp. Pharmacol. Physiol. 35, 670–678 (2008)
Ludbrook, J., Dudley, H.A.F.: Issues in biomedical statistics: analyzing 2 × 2 tables of frequencies. Aust. N. Z. J. Surg. 64, 780–787 (1994)
Ludbrook, J., Dudley, H.A.F.: Issues in biomedical statistics: statistical inference. Aust. N. Z. J. Surg. 64, 630–636 (1994)
Ludbrook, J., Dudley, H.A.F.: Why permutation tests are superior to t and F tests in biomedical research. Am. Stat. 52, 127–132 (1998)
Ludbrook, J., Dudley, H.A.F.: Discussion of “Why permutation tests are superior to t and F tests in biomedical research” by J. Ludbrook and H.A.F. Dudley. Am. Stat. 54, 87 (2000)
Lunneborg, C.E.: Data Analysis by Resampling: Concepts and Applications. Duxbury, Pacific Grove (2000)
Maclure, M., Willett, W.C.: Misinterpretation and misuse of the kappa statistic. Am. J. Epidemiol. 126, 161–169 (1987)
Manly, B.F.J.: Randomization and Monte Carlo Methods in Biology. Chapman & Hall, London (1991)
Manly, B.F.J.: Randomization and Monte Carlo Methods in Biology, 2nd edn. Chapman & Hall, London (1997)
Manly, B.F.J.: Randomization, Bootstrap and Monte Carlo Methods in Biology, 3rd edn. Chapman & Hall/CRC, Boca Raton (2007)
Manly, B.F.J., Francis, R.I.C.: Analysis of variance by randomization when variances are unequal. Aust. N. Z. J. Stat. 41, 411–429 (1999)
Mann, H.B., Whitney, D.R.: On a test of whether one of two random variables is stochastically larger than the other. Ann. Math. Stat. 18, 50–60 (1947)
Margolin, B.H., Light, R.J.: An analysis of variance for categorical data, II: small sample comparisons with chi square and other competitors. J. Am. Stat. Assoc. 69, 755–764 (1974)
Mathew, T., Nordström, K.: Least squares and least absolute deviation procedures in approximately linear models. Stat. Probab. Lett. 16, 153–158 (1993)
Maxim, P.S.: Quantitative Research Methods in the Social Sciences. Oxford, New York (1999)
Maxwell, S.E., Camp, C.J., Arvey, R.D.: Measures of strength of association: a comparative examination. J. Appl. Psychol. 66, 525–534 (1981)
May, R.B., Hunter, M.A.: Some advantages of permutation tests. Can. Psychol. 34, 401–407 (1993)
May, S.M.: Modelling observer agreement: an alternative to kappa. J. Clin. Epidemiol. 47, 1315–1324 (1994)
McCarthy, M.D.: On the application of the z-test to randomized blocks. Ann. Math. Stat. 10, 337–359 (1939)
McGrath, R.E., Meyer, G.J.: When effect sizes disagree: the case of r and d. Psychol. Methods 11, 386–401 (2006)
McHugh, R.B., Mielke, P.W.: Negative variance estimates and statistical dependence in nested sampling. J. Am. Stat. Assoc. 63, 1000–1003 (1968)
McLean, J.E., Ernest, J.M.: The role of statistical significance testing in educational research. J. Health Soc. Behav. 5, 15–22 (1998)
McNemar, Q.: Note on the sampling error of the differences between correlated proportions and percentages. Psychometrika 12, 153–157 (1947)
McQueen, G.: Long-horizon mean-reverting stock priced revisited. J. Financ. Quant. Anal. 27, 1–17 (1992)
Mehta, C.R., Patel, N.R.: Algorithm 643: FEXACT. A FORTRAN subroutine for Fisher’s exact test on unordered r × c contingency tables. ACM Trans. Math. Softw. 12, 154–161 (1986)
Mehta, C.R., Patel, N.R.: A hybrid algorithm for Fisher’s exact test in unordered r × c contingency tables. Commun. Stat. Theory Methods 15, 387–403 (1986)
Mehta, C.R., Patel, N.R., Gray, R.: On computing an exact confidence interval for the common odds ratio in several 2 × 2 contingency tables. J. Am. Stat. Assoc. 80, 969–973 (1985)
Metropolis, N., Ulam, S.: The Monte Carlo method. J. Am. Stat. Assoc. 44, 335–341 (1949)
Meyer, G.J.: Assessing reliability: critical corrections for a critical examination of the Rorschach comprehensive system. Psychol. Assess. 9, 480–489 (1997)
Micceri, T.: The unicorn, the normal curve, and other improbable creatures. Psychol. Bull. 105, 156–166 (1989)
Mielke, P.W.: Asymptotic behavior of two-sample tests based on powers of ranks for detecting scale and location alternatives. J. Am. Stat. Assoc. 67, 850–854 (1972)
Mielke, P.W.: Squared rank test appropriate to weather modification cross-over design. Technometrics 16, 13–16 (1974)
Mielke, P.W.: Convenient beta distribution likelihood techniques for describing and comparing meteorological data. J. Appl. Meterol. 14, 985–990 (1975)
Mielke, P.W.: Meteorological applications of permutation techniques based on distance functions. In: Krishnaiah, P.R., Sen, P.K. (eds.) Handbook of Statistics, vol. IV, pp. 813–830. North-Holland, Amsterdam (1984)
Mielke, P.W.: Geometric concerns pertaining to applications of statistical tests in the atmospheric sciences. J. Atmos. Sci. 42, 1209–1212 (1985)
Mielke, P.W.: Non-metric statistical analyses: some metric alternatives. J. Stat. Plan Inference 13, 377–387 (1986)
Mielke, P.W.: The application of multivariate permutation methods based on distance functions in the earth sciences. Earth Sci. Rev. 31, 55–71 (1991)
Mielke, P.W., Berry, K.J.: An extended class of permutation techniques for matched pairs. Commun. Stat. Theory Methods 11, 1197–1207 (1982)
Mielke, P.W., Berry, K.J.: Asymptotic clarifications, generalizations, and concerns regarding an extended class of matched pairs tests based on powers of ranks. Psychometrika 48, 483–485 (1983)
Mielke, P.W., Berry, K.J.: Cumulant methods for analyzing independence of r-way contingency tables and goodness-of-fit frequency data. Biometrika 75, 790–793 (1988)
Mielke, P.W., Berry, K.J.: Permutation tests for common locations among samples with unequal variances. J. Educ. Behav. Stat. 19, 217–236 (1994)
Mielke, P.W., Berry, K.J.: Nonasymptotic inferences based on Cochran’s Q test. Percept. Mot. Skill 81, 319–322 (1995)
Mielke, P.W., Berry, K.J.: Permutation-based multivariate regression analysis: the case for least sum of absolute deviations regression. Ann. Oper. Res. 74, 259–268 (1997)
Mielke, P.W., Berry, K.J.: Permutation covariate analyses of residuals based on Euclidean distance. Psychol. Rep. 81, 795–802 (1997)
Mielke, P.W., Berry, K.J.: Euclidean distance based permutation methods in atmospheric science. Data Min. Knowl. Disc. 4, 7–27 (2000)
Mielke, P.W., Berry, K.J.: Data-dependent analyses in psychological research. Psychol. Rep. 91, 1225–1234 (2002)
Mielke, P.W., Berry, K.J.: Permutation Methods: A Distance Function Approach, 2nd edn. Springer, New York (2007)
Mielke, P.W., Berry, K.J.: A note on Cohen’s weighted kappa coefficient of agreement with linear weights. Stat. Methodol. 6, 439–446 (2009)
Mielke, P.W., Iyer, H.K.: Permutation techniques for analyzing multi-response data from randomized block experiments. Commun. Stat. Theory Methods 11, 1427–1437 (1982)
Mielke, P.W., Berry, K.J., Johnson, E.S.: Multi-response permutation procedures for a priori classifications. Commun. Stat. Theory Methods 5, 1409–1424 (1976)
Mielke, P.W., Berry, K.J., Brier, G.W.: Application of multi-response permutation procedures for examining seasonal changes in monthly mean sea-level pressure patterns. Mon. Weather Rev. 109, 120–126 (1981)
Mielke, H.W., Anderson, J.C., Berry, K.J., Mielke, P.W., Chaney, R.L., Leech, M.: Lead concentrations in inner-city soils as a factor in the child lead problem. Am. J. Public Health 73, 1366–1369 (1983)
Mielke, P.W., Berry, K.J., Landsea, C.W., Gray, W.M.: Artificial skill and validation in meteorological forecasting. Weather Forecast. 11, 153–169 (1996)
Mielke, P.W., Berry, K.J., Neidt, C.O.: A permutation test for multivariate matched-pairs analyses: comparisons with Hotelling’s multivariate matched-pairs T 2 test. Psychol. Rep. 78, 1003–1008 (1996)
Mielke, P.W., Berry, K.J., Johnston, J.E.: A FORTRAN program for computing the exact variance of weighted kappa. Percept. Mot. Skill 101, 468–472 (2005)
Mielke, P.W., Berry, K.J., Johnston, J.E.: The exact variance of weighted kappa with multiple raters. Psychol. Rep. 101, 655–660 (2007)
Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling programs for multiway contingency tables with fixed marginal frequency totals. Psychol. Rep. 101, 18–24 (2007)
Mielke, P.W., Berry, K.J., Johnston, J.E.: Resampling probability values for weighted kappa with multiple raters. Psychol. Rep. 102, 606–613 (2008)
Mielke, P.W., Berry, K.J., Johnston, J.E.: Robustness without rank order statistics. J. Appl. Stat. 38, 207–214 (2011)
Minkowski, H.: Über die positiven quadratishen formen und über kettenbruchähnliche algorithmen. Crelle’s J (J. Reine Angew. Math.) 107, 278–297 (1891). [Also available in H. Minkowski, Gesammelte Abhandlungen, vol. 1, AMS Chelsea, New York, 1967]
Mitchell, C., Hartmann, D.P.: A cautionary note on the use of omega squared to evaluate the effectiveness of behavioral treatments. Behav. Assess. 3, 93–100 (1981)
Mood, A.M.: On the asymptotic efficiency of certain nonparametric two-sample tests. Ann. Math. Stat. 25, 514–522 (1954)
Moses, L.E.: Statistical theory and research design. Ann. Rev. Psychol. 7, 233–258 (1956)
Murphy, K.R., Cleveland, J.: Understanding Performance Appraisal: Social, Organizational, and Goal-Based Perspectives. Sage, Thousand Oaks (1995)
Myers, J.L., Well, A.D.: Research Design and Statistical Analysis. HarperCollins, New York (1991)
Nanda, D.N.: Distribution of the sum of roots of a determinantal equation. Ann. Math. Stat. 21, 432–439 (1950)
Neave, H.R., Worthington, P.L.: Distribution-Free Tests. Unwin Hyman, London (1988)
Newson, R.: Parameters behind “nonparametric” statistics: Kendall’s tau, Somers’ D and median differences. Stata J. 2, 45–64 (2002)
Neyman, J., Pearson, E.S.: On the use and interpretation of certain test criteria for purposes of statistical inference: part I. Biometrika 20A, 175–240 (1928)
Neyman, J., Pearson, E.S.: On the use and interpretation of certain test criteria for purposes of statistical inference: part II. Biometrika 20A, 263–294 (1928)
Nix, T.W., Barnette, J.J.: The data analysis dilemma: Ban or abandon. A review of null hypothesis significance testing. Res. Schools 5, 3–14 (1998)
Nix, T.W., Barnette, J.J.: A review of hypothesis testing revisited: Rejoinder to Thompson, Knapp, and Levin. Res. Schools 5, 55–57 (1998)
O’Boyle, Jr., E., Aguinis, H.: The best and the rest: revisiting the norm of normality of individual performance. Percept. Psychophys. 65, 79–119 (2012)
Okamoto, D.: Letter to the editor: does it work for coffee? Significance 10, 45–46 (2013)
Olds, E.G.: Distribution of sums of squares of rank differences for small numbers of individuals. Ann. Math. Stat. 9, 133–148 (1938)
Olejnik, S., Algina, J.: Measures of effect size for comparative studies: applications, interpretations, and limitations. Contemp. Educ. Psychol. 25, 241–286 (2000)
Olson, C.L.: On choosing a test statistic in multivariate analysis of variance. Psychol. Bull. 83, 579–586 (1976)
Olson, C.L.: Practical considerations in choosing a MANOVA test statistic: a rejoinder to Stevens. Psychol. Bull. 86, 1350–1352 (1979)
Osgood, C.E., Suci, G., Tannenbaum, P.: The Measurement of Meaning. University of Illinois Press, Urbana (1957)
Overall, J.E., Spiegel, D.K.: Concerning least squares analysis of experimental data. Psychol. Bull. 72, 311–322 (1969)
Pagano, R.R.: Understanding Statistics in the Behavioral Sciences, 6th edn. Wadsworth, Pacific Grove (2001)
Pearson, K.: Contributions to the mathematical theory of evolution. Proc. R. Soc. Lond. 54, 329–333 (1893)
Pearson, K.: Contributions to the mathematical theory of evolution, II. Skew variation in homogeneous material. Philos. Trans. R. Soc. Lond. A 186, 343–414 (1895)
Pearson, K.: Mathematical contributions to the theory of evolution, XIII. On the theory of contingency and its relation to association and normal correlation. In: Drapers’ Company Research Memoirs, Biometric Series I, pp. 1–35. Cambridge University Press, Cambridge (1904)
Pearson, E.S.: Untitled. Nature 123, 866–867 (1929). [Review by E.S. Pearson of the second edition of R.A. Fisher’s Statistical Methods for Research Workers]
Pearson, K., Heron, D.: On theories of association. Biometrika 9, 159–315 (1913)
Pfaffenberger, R., Dinkel, J.: Absolute deviations curve-fitting: an alternative to least squares. In: David, H.A. (ed.) Contributions to Survey Sampling and Applied Statistics, pp. 279–294. Academic Press, New York (1978)
Picard, R.: Randomization and design: II. In: Feinberg, S.E., Hinkley, D.V. (eds.) R. A. Fisher: An Appreciation, pp. 46–58. Springer, Heidelberg (1980)
Pillai, K.C.S.: Some new test criteria in multivariate analysis. Ann. Math. Stat. 26, 117–121 (1955)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations. Suppl. J. R. Stat. Soc. 4, 119–130 (1937)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: II. The correlation coefficient test. Suppl. J. R. Stat. Soc. 4, 225–232 (1937)
Pitman, E.J.G.: Significance tests which may be applied to samples from any populations: III. The analysis of variance test. Biometrika 29, 322–335 (1938)
Randles, R.H., Wolfe, D.A.: Introduction to the Theory of Nonparametric Statistics. Wiley, New York (1979)
Raveh, A.: On measures of monotone association. Am. Stat. 40, 117–123 (1986)
Reinhart, A.: Statistics Done Wrong: The Woefully Complete Guide. No Starch Press, San Francisco (2015)
Rice, J., White, J.: Norms for smoothing and estimation. SIAM Rev. 6, 243–256 (1964)
Ricketts, C., Berry, J.S.: Teaching statistics through resampling. Teach. Stat. 16, 41–44 (1994)
Roberts, J.K., Henson, R.K.: Correcting for bias in estimating effect sizes. Educ. Psychol. Meas. 62, 241–253 (2002)
Robinson, W.S.: Ecological correlations and the behavior of individuals. Am. Soc. Rev. 15, 351–357 (1950). [Reprinted in Int. J. Epidemiol. 38, 337–341 (2009)]
Robinson, W.S.: The statistical measurement of agreement. Am. Sociol. Rev. 22, 17–25 (1957)
Robinson, W.S.: The geometric interpretation of agreement. Am. Sociol. Rev. 24, 338–345 (1959)
Rosenberg, B., Carlson, D.: A simple approximation of the sampling distribution of least absolute residuals regression estimates. Commun. Stat. Simul. Comput. 6, 421–438 (1977)
Rosenthal, R., Rosnow, R.L., Rubin, D.B.: Contrasts and Effect Sizes in Behavioral Research: A Correlational Approach. Cambridge University Press, Cambridge (2000)
Rouanet, H., Lépine, D.: Comparison between treatments in a repeated measures design: ANOVA and multivariate methods. Br. J. Math. Stat. Psychol. 23, 147–164 (1970)
Rousseeuw, P.J.: Least median of squares regression. J. Am. Stat. Assoc. 79, 421–438 (1984)
Routledge, R.D.: Resolving the conflict over Fisher’s exact test. Can. J. Stat. 20, 201–209 (1992)
Roy, S.N.: On a heuristic method of test construction and its use in multivariate analysis. Ann. Math. Stat. 24, 220–238 (1953)
Roy, S.N.: Some Aspects of Multivariate Analysis. Wiley, New York (1957)
Saal, F.E., Downey, R.G., Lahey, M.A.: Rating the ratings: assessing the quality of rating data. Psychol. Bull. 88, 413–428 (1980)
Salama, I.A., Quade, D.: A note on Spearman’s footrule. Commun. Stat. Simul. Comput. 19, 591–601 (1990)
Salsburg, D.: The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Holt, New York (2001)
Särndal, C.E.: A comparative study of association measures. Psychometrika 39, 165–187 (1974)
Satterthwaite, F.E.: An approximate distribution of estimates of variance components. Biom. Bull. 2, 110–114 (1946)
Scheffé, H.: Statistical inference in the non-parametric case. Ann. Math. Stat. 14, 305–332 (1943)
Scheffé, H.: The Analysis of Variance. Wiley, New York (1959)
Schmidt, F.L., Johnson, R.H.: Effect of race on peer ratings in an industrial situation. J. Appl. Psychol. 57, 237–241 (1973)
Schuster, C.: A note on the interpretation of weighted kappa and its relations to other rater agreement statistics for metric scales. Educ. Psychol. Meas. 64, 243–253 (2004)
Scott, W.A.: Reliability of content analysis: the case of nominal scale coding. Public Opin. Q. 19, 321–325 (1955)
Senn, S.: Fisher’s game with the devil. Stat. Med. 13, 217–230 (1994). [Publication of a paper presented at the Statisticians in the Pharmaceutical Industry (PSI) annual conference held in Sept 1991 in Bristol, England]
Senn, S.: Tea for three: of infusions and inferences and milk in first. Significance 9, 30–33 (2012)
Senn, S.: Response to “Tea break” by S. Springate. Significance 10, 46 (2013)
Sheynin, O.B.: R. J. Boscovich’s work on probability. Arch. Hist. Exact Sci. 9, 306–324 (1973)
Shrout, P.E., Fleiss, J.L.: Intraclass correlations: uses in assessing rater relaibility. Psychol. Bull. 86, 420–428 (1979)
Shrout, P.E., Spitzer, R.L., Fleiss, J.L.: Quantification of agreement in psychiatric diagnosis revisited. Arch. Gen. Psychiatry 44, 172–177 (1987)
Siegel, S., Castellan, N.J.: Nonparametric Statistics for the Behavioral Sciences, 2nd edn. McGraw-Hill, New York (1988)
Siegel, S., Tukey, J.W.: A nonparametric sum of ranks procedure for relative spread in unpaired samples. J. Am. Stat. Assoc. 55, 429–445 (1960). [Corrigendum: J. Am. Stat. Assoc. 56, 1005 (1961)]
Siegfried, T.: Odds are, it’s wrong. Sci. News 177, 26–29 (2010)
Snedecor, G.W.: Calculation and Interpretation of Analysis of Variance and Covariance. Collegiate Press, Ames (1934)
Snyder, P., Lawson, S.: Evaluating results using corrected and uncorrected effect size estimates. J. Exp. Educ. 61, 334–349 (1993)
Somers, R.H.: A new asymmetric measure of association for ordinal variables. Am. Sociol. Rev. 27, 799–811 (1962)
Spearman, C.E.: The proof and measurement of association between two things. Am. J. Psychol. 15, 72–101 (1904)
Spearman, C.E.: ‘Footrule’ for measuring correlation. Br. J. Psychol. 2, 89–108 (1906)
Spitznagel, E.L., Helzer, J.E.: A proposed solution to the base rate problem in the kappa statistic. Arch. Gen. Psychiatry 42, 725–728 (1985)
Springate, S.: Tea break. Significance 10, 45–46 (2013)
Stark, R., Roberts, I.: Contemporary Social Research Methods. Micro-Case, Bellevue (1996)
Stevens, J.P.: Applied Multivariate Statistics for the Social Sciences. Erlbaum, Hillsdale (1986)
Stevens, J.P.: Intermediate Statistics: A Modern Approach. Erlbaum, Hillsdale (1990)
Still, A.W., White, A.P.: The approximate randomization test as an alternative to the F test in analysis of variance. Br. J. Math. Stat. Psychol. 34, 243–252 (1981)
Stuart, A.: The estimation and comparison of strengths of association in contingency tables. Biometrika 40, 105–110 (1953)
“Student”: The probable error of a mean. Biometrika 6, 1–25 (1908). [“Student” is a nom de plume for William Sealy Gosset]
Susskind, E.C., Howland, E.W.: Measuring effect magnitude in repeated measures ANOVA designs: implications for gerontological research. J. Gerontol. 35, 867–876 (1980)
Tabachnick, B.G., Fidell, L.S.: Using Multivariate Statistics, 5th edn. Pearson, Boston (2007)
Taha, M.A.H.: Rank test for scale parameter for asymmetrical one-sided distributions. Publ. Inst. Stat. Univ. Paris 13, 169–180 (1964)
Taylor, L.D.: Estimation by minimizing the sum of absolute errors. In: Zarembka, P. (ed.) Frontiers in Econometrics, pp. 169–190. Academic Press, New York (1974)
Tedin, O.: The influence of systematic plot arrangements upon the estimate of error in field experiments. J. Agric. Sci. 21, 191–208 (1931)
Thompson, D.W.: On Growth and Form: The Complete Revised Edition. Dover, New York (1992)
Thompson, W.L.: 402 citations questioning the indiscriminate use of null hypothesis significance tests in observational studies. http://www.warnercnr.colostate.edu/~anderson/thompson1.html (2001). Accessed 18 June 2015
Thompson, W.L.: Problems with the hypothesis testing approach. http://www.warnercnr.colostate.edu/~gwhite/fw663/testing.pdf (2001). Accessed 18 June 2015
Thompson, W.D., Walter, S.D.: A reappraisal of the kappa coefficient. J. Clin. Epidemiol. 41, 949–958 (1988)
Trafimow, D.: Editorial. Basic Appl. Soc. Psychol. 36, 1–2 (2014)
Trafimow, D., Marks, M.: Editorial. Basic Appl. Soc. Psychol. 37, 1–2 (2015)
Tschuprov, A.A.: Principles of the Mathematical Theory of Correlation. Hodge, London (1939). [Translated by M. Kantorowitsch]
Tukey, J.W.: Data analysis and behavioral science (1962). [Unpublished manuscript]
Tukey, J.W.: The future of data analysis. Ann. Math. Stat. 33, 1–67 (1962)
Tukey, J.W.: Randomization and re-randomization: the wave of the past in the future. In: Statistics in the Pharmaceutical Industry: Past, Present and Future. Philadelphia Chapter of the American Statistical Association (1988). [Presented at a Symposium in Honor of Joseph L. Ciminera held in June 1988 at Philadelphia, Pennsylvania]
Umesh, U.N.: Predicting nominal variable relationships with multiple response. J. Forecast. 14, 585–596 (1995)
Umesh, U.N., Peterson, R.A., Sauber, M.H.: Interjudge agreement and the maximum value of kappa. Educ. Psychol. Meas. 49, 835–850 (1989)
Ury, H.K., Kleinecke, D.C.: Tables of the distribution of Spearman’s footrule. J. R. Stat. Soc.: Ser. C: Appl. Stat. 28, 271–275 (1979)
van der Reyden, D.: A simple statistical significance test. Rhod. Agric. J. 49, 96–104 (1952)
Vanbelle, S., Albert, A.: A note on the linearly weighted kappa coefficient for ordinal scales. Stat. Methodol. 6, 157–163 (2008)
Vaughan, G.M., Corballis, M.C.: Beyond tests of significance: estimating strength of effects in selected ANOVA designs. Psychol. Bull. 79, 391–395 (1969)
von Eye, A., von Eye, M.: On the marginal dependency of Cohen’s κ. Eur. Pychol. 13, 305–315 (2008)
Wald, A., Wolfowitz, J.: An exact test for randomness in the non-parametric case based on serial correlation. Ann. Math. Stat. 14, 378–388 (1943)
Wallis, W.A.: The correlation ratio for ranked data. J. Am. Stat. Assoc. 34, 533–538 (1939)
Watnik, M.: Early computational statistics. J. Comput. Graph. Stat. 20, 811–817 (2011)
Watterson, I.G.: Nondimensional measures of climate model performance. Int. J. Climatol. 16, 379–391 (1996)
Welch, B.L.: The specification of rules for rejecting too variable a product, with particular reference to an electric lamp problem. Suppl. J. R. Stat. Soc. 3, 29–48 (1936)
Welch, B.L.: On the z-test in randomized blocks and Latin squares. Biometrika 29, 21–52 (1937)
Welch, B.L.: The significance of the difference between two means when the population variances are unequal. Biometrika 29, 350–362 (1938)
Welch, B.L.: On the comparison of several mean values: an alternative approach. Biometrika 38, 330–336 (1951)
Welkowitz, J., Ewen, R.B., Cohen, J.: Introductory Statistics for the Behavioral Sciences, 5th edn. Harcourt Brace, Orlando (2000)
Wherry, R.J.: A new formula for predicting the shrinkage of the coefficient of multiple correlation. Ann. Math. Stat. 2, 440–457 (1931)
Whitehurst, G.J.: Interrater agreement for journal manuscript reviews. Am. Psychol. 39, 22–28 (1984)
Whitfield, J.W.: Rank correlation between two variables, one of which is ranked, the other dichotomous. Biometrika 34, 292–296 (1947)
Wickens, T.D.: Multiway Contingency Tables Analysis for the Social Sciences. Erlbaum, Hillsdale (1989)
Wilcox, R.R.: Statistics for the Social Sciences. Academic Press, San Diego (1996)
Wilcox, R.R.: Applying Contemporary Statistical Techniques. Academic Press, San Diego (2003)
Wilcox, R.R., Muska, J.: Measuring effect size: a non-parametric analgue of \(\hat{\omega }^{2}\). Br. J. Math. Stat. Psychol. 52, 93–110 (1999)
Wilcoxon, F.: Individual comparisons by ranking methods. Biom. Bull. 1, 80–83 (1945)
Wilkinson, L.: Statistical methods in psychology journals: guidelines and explanations. Am. Psychol. 54, 594–604 (1999)
Wilks, S.S.: Certain generalizations in the analysis of variance. Biometrika 24, 471–494 (1932)
Wilson, H.G.: Least squares versus minimum absolute deviations estimation in linear models. Decis. Sci. 9, 322–325 (1978)
Yates, F.: Contingency tables involving small numbers and the χ 2 test. Suppl. J. R. Stat. Soc. 1, 217–235 (1934)
Yule, G.U.: On the association of attributes in statistics: with illustrations from the material childhood society. Philos. Trans. R. Soc. Lond. 194, 257–319 (1900)
Yule, G.U.: On the methods of measuring association between two attributes. J. R. Stat. Soc. 75, 579–652 (1912). [Originally a paper read before the Royal Statistical Society on 23 April 1912]
Zwick, R.: Another look at interrater agreement. Psychol. Bull. 103, 374–378 (1988)
Author information
Authors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Berry, K.J., Mielke, P.W., Johnston, J.E. (2016). Introduction. In: Permutation Statistical Methods. Springer, Cham. https://doi.org/10.1007/978-3-319-28770-6_1
Download citation
DOI: https://doi.org/10.1007/978-3-319-28770-6_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-28768-3
Online ISBN: 978-3-319-28770-6
eBook Packages: Mathematics and StatisticsMathematics and Statistics (R0)