Comparison of the Performance of Nonparametric and Parametric MANOVA Test Statistics when Assumptions Are Violated
Abstract
Abstract. Multivariate analysis of variance (MANOVA) is a useful tool for social scientists because it allows for the comparison of response-variable means across multiple groups. MANOVA requires that the observations are independent, the response variables are multivariate normally distributed, and the covariance matrix of the response variables is homogeneous across groups. When the assumptions of normality and homogeneous covariance matrices are not met, past research has shown that the type I error rate of the standard MANOVA test statistics can be inflated while their power can be attenuated. The current study compares the performance of a nonparametric alternative to one of the standard parametric test statistics when these two assumptions are not met. Results show that when the assumption of homogeneous covariance matrices is not met, the nonparametric approach has a lower type I error rate and higher power than the most robust parametric statistic. When the assumption of normality is untenable, the parametric statistic is robust, and slightly outperforms the nonparametric statistic in terms of type I error rate and power.
References
Algina, J. , Oshima, T.C. , Tang, K.L. (1991). Robustness of Yao's, James', and Johansen's tests under variance-covariance heteroscedasticity and nonnormality. Journal of the American Statistical Association, 86, 457– 460Cohen, J. (1988). Statistical power analysis for the behavioral sciences . Hillsdale, NJ: ErlbaumConover, W.J. (1999). Practical nonparametric statistics . New York: WileyErdfelder, E. (1981). Multivariate Rangvarianzanalyse: Ein nonparametrisches Analogon zur ein- und mehrfaktoriellen MANOVA . [Multivariate rank variance analysis: A nonparametric analogue for single and multivariate MANOVAs] Trierer Psychologische Berichte, 8(12), Trier, Germany: Fachbereich 1 - Psychologie der Universität TrierEveritt, B.S. (1979). A Monte Carlo investigation of the robustness of Hotelling's one and two sample T2 tests. Journal of the American Statistical Association, 74, 48– 51Glass, G.V. (1972). Consequences of failure to meet assumptions underlying the analysis of variance and covariance. Review of Educational Research, 42, 237– 288Hakstian, A.R. , Roed, J.C. , Lind, J.C. (1979). Two-sample T2 procedure and the assumption of homogeneous covariance matrices. Psychological Bulletin, 86, 1255– 1263Headrick, T.C. , Sawilosky, S.S. (1999). Simulating correlated multivariate nonnormal distributions: Extending the Fleishman power method. Psychometrika, 64, 25– 35Hoenig, J.M. , Heisey, D.M. (2001). The abuse of power: The pervasive fallacy of power calculations for data analysis. The American Statistician, 55, 19– 24Ito, K. , Schull, W.J. (1964). On the robustness of the T2 test in multivariate analysis of variance when variance-covariance matrices are not equal. Biometrika, 51, 71– 82Ittenbach, R.F. , Chayer, D.E. , Bruininks, R.H. , Thrulow, M.L. , Beirne-Smith, M. (1993). Adjustment of young adults with mental retardation in community settings: Comparison of parametric and nonparametric statistical techniques. American Journal of Mental Retardation, 97, 607– 615Katz, B.M. , McSweeney, M. (1980). A multivariate Kruskal-Wallis test with post hoc procedures. Multivariate Behavioral Research, 15, 281– 297Kim, S. , Olejnik, S. (2004). Bias and precision of multivariate effect size measures of association for a fixed-effect analysis of variance model . Paper presented a the annual meeting of the American Educational Research Association, San Diego, CA April, 2004Marascuilo, L.A. , McSweeney, M. (1977). Nonparametric and distribution-free methods for the social sciences . Monterey, CA: Brooks/ColeMardia, K.V. (1971). The effect of nonnormality on some multivariate tests and robustness to nonnormality in the linear model. Biometrika, 58(1), 105– 121Olson, C.L. (1979). Practical considerations in choosing a MANOVA test statistic: A rejoinder to Stevens. Psychological Bulletin, 86, 1350– 1352Olson, C.L. (1976). On choosing a test statistic in multivariate analysis of variance. Psychological Bulletin, 83, 579– 586Olson, C.L. (1974). Comparative robustness of six tests in multivariate analysis of variance. Journal of the American Statistical Association, 69, 894– 908Puri, M.L. , Sen, P.K. (1971). Nonparametric methods in multivariate analysis . New York: WileySheehan-Holt, J.K. (1998). MANOVA simultaneous test procedures: The power and robustness of restricted multivariate contrasts. Educational and Psychological Measurement, 58, 861– 881Srivastava, D.K. , Mudholkar, G.S. (2001). Trimmed T2: A robust analog of Hotelling's T2. Journal of Statistical Planning and Inference, 97, 343– 358Stevens, J. (1996). Applied multivariate statistics for the social sciences . Mahwah, NJ: Lawrence Erlbaum AssociatesTabachnick, B.G. , Fidell, L.S. (2001). Using multivariate statistics . New York: Harper CollinsTiku, M.L. , Balakrishnan, N. (1988). Robust Hotelling-type T2 statistics based on the modified maximum likelihood estimators. Communications in Statistics - Theory and Methods, 17, 1789– 1810Zwick, R. (1985). Nonparametric one-way multivariate analysis of variance: A computational approach based on the Pillai-Bartlett trace. Psychological Bulletin, 97, 148– 152
