Other Multiple Comparison Methods

A complete data analysis often requires more than a simple overall F test or a limited number of planned comparisons. Many important discoveries are “after the fact”—unanticipated relationships found in the data. Such relationships cannot be rested by planned comparisons; the choice of a comparison on the basis of apparent differences among the obtained means would introduce a strong bias in favor of rejecting the null hypothesis. Other techniques for making multiple comparisons exist, but they have very low power; in many cases, they will not find a significant difference unless it is large enough to be obvious without a test.