Abstract
The most popular form of artificial neural network, feedforward networks with sigmoidal activation functions, and a new statistical technique, multivariate adaptive regression splines (MARS) can both be classified as nonlinear, nonparametric function estimation techniques, and both show great promise for fitting general nonlinear multivariate functions. In comparing the two methods on a variety of test problems, we find that MARS is in many cases both more accurate and much faster than neural networks. In addition, MARS is interpretable due to the choice of basic functions which make up the final predictive equation. This suggests that MARS could be used on many of the applications where neural networks are currently being used.
However, MARS exhibits problems in choosing among predictor variables when multicollinearity is present. Due to their redundant architecture, neural networks, however, do not share this problem, and are better able to predict in this situation. To improve the ability of MARS to deal with multicollinearity, we first use principal components to reduce the dimensionality of the input variables before invoking MARS. Using data from a polymer production run, we find that the resulting model retains the interpretability and improves the accuracy of MARS in the multicollinear setting.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Barron, A. R. “Approximation and Estimation Bounds for Artificial Neural Networks.”, in press, Machine Learning (1992).
Chambers, J. M. and Hastie, T. J., “Statistical Models in S”, Wadsworth & Brooks/Cole, Pacific Grove, California, 1992.
De Veaux, R. D., Psichogios, D. C., and Ungar, L. H. “A Comparison of Two Non-Parametric Estimation Schemes: MARS and Neural Networks.” Comp. Chem. Engng. 17:8 (1993) 819–837.
Friedman, J. H. “Multivariate adaptive regression splines.” The Annals of Statistics 19:1 (1991) 1–141.
Holcomb, T. R., and Morari, M. “PLS/Neural Networks.” Comp. Chem. Engng. 16:4, (1992) 393–411.
Psichogios, D. C., De Veaux, R. D., and Ungar, L. H. “Non-Parametric System Identification: A Comparison of MARS and Neural Networks.” ACC TA4 (1992) 1436–1440.
Qin, S. J., and McAvoy, T. J. “Nonlinear PLS Modeling Using Neural Networks.” Comp. Chem. Engng. 16:4, (1992) 379–391.
Ripley, B. D. “Statistical Aspects of Neural Networks.” Invited lectures for SemStat, Sandbjerg, Denmark, 25–30 April 1992.
Rumelhart, D., Hinton, G., and Williams, R. “Learning Internal Representations by Error Propagation.” Parallel Distributed Processing: Explorations in the Microstructures of Cognition, Vol 1: Foundations Cambridge: MIT Press (1986), 318–362.
Skulic, S., and Kowalski, B. R. “MARS: A Tutorial.” J. Chemometrics 6, (1992).
Skulic, S., and Kowalski, B. R. “Nonlinear Multivariate Calibration Methods Combined with Dimensionality Reduction.” submitted to J. Chemometrics (1992).
Wold, S., Kettaneh-Wold, N., and Skagerberg, B. “Nonlinear PLS Modeling.” Chemometrics and Intelligent Laboratory Systems, 7 (1989) 53–65.
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 1994 Springer-Verlag New York, Inc.
About this paper
Cite this paper
De Veaux, R.D., Ungar, L.H. (1994). Multicollinearity: A tale of two nonparametric regressions. In: Cheeseman, P., Oldford, R.W. (eds) Selecting Models from Data. Lecture Notes in Statistics, vol 89. Springer, New York, NY. https://doi.org/10.1007/978-1-4612-2660-4_40
Download citation
DOI: https://doi.org/10.1007/978-1-4612-2660-4_40
Publisher Name: Springer, New York, NY
Print ISBN: 978-0-387-94281-0
Online ISBN: 978-1-4612-2660-4
eBook Packages: Springer Book Archive