Small sample properties of tests on homogeneity in one—way Anova and Meta—analysis

Abstract

In the present Monte Carlo study, the empirical Type I error properties and power of several statistics for testing the homogeneity hypothesis in a one—way classification are examined in the case of small sample sizes. We compared these tests under several scenarios: normal populations under heterogeneous variances, nonnormal populations under homogeneous variances, nonnormal populations under heterogeneous variances, balanced and unbalanced sample sizes, and increasing number of populations. Overall, none of the tests considered is uniformly dominating the others. Under normality and variance heterogeneity, the Brown—Forsythe and the Welch test perform well over a wide range of parameter configurations, the modified Brown-Forsythe test by Mehrotra keeps generally the level, but other tests may also perform well, depending on the constellation of the parameters under study. The Welch test becomes liberal when the sample sizes are small and the number of populations is large. We propose a modified version of Welch’s test that keeps the nominal level in these cases. With the understanding that methods are unacceptable if they have Type I error rates that are too high, only the testing procedure associated with the modified Brown-Forsythe test can be recommended both for normal and nonnormal data. Under normality, the modified Welch test can also be recommended.