Abstract
A general approach for fitting a model to a data matrix by weighted least squares (WLS) is studied. This approach consists of iteratively performing (steps of) existing algorithms for ordinary least squares (OLS) fitting of the same model. The approach is based on minimizing a function that majorizes the WLS loss function. The generality of the approach implies that, for every model for which an OLS fitting algorithm is available, the present approach yields a WLS fitting algorithm. In the special case where the WLS weight matrix is binary, the approach reduces to missing data imputation.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Bailey, R. A., & Gower, J. C. (1990). Approximating a symmetric matrix.Psychometrika, 55, 665–675.
Bijleveld, C. & de Leeuw, J. (1991). Fitting longitudinal reduced rank regression models by alternating least squares.Psychometrika, 56, 443–447.
Bollen, K. A. (1989).Structural equations with latent variables. New York: Wiley.
Browne, M. W. (1984). Asymptotically distribution-free methods for the analysis of covariance structures.British Journal of Mathematical and Statistical Psychology, 37, 62–83.
Carroll, J. D., De Soete, G., & Pruzansky, S. (1989). Fitting of the latent class model via iteratively reweighted least squares candecomp with nonnegativity constraints. In R. Coppi & S. Bolasco (Eds.),Multiway data analysis (pp. 463–472). Amsterdam: Elsevier Science Publishers.
Cliff, N. (1966). Orthogonal rotation to congruence.Psychometrika, 31, 33–42.
Commandeur, J. J. F. (1991). Matching configurations. Leiden: DSWO Press.
de Leeuw, J. & Heiser, W. (1980). Multidimensional scaling with restrictions on the configuration. In P. R. Krishnaiah (Ed.),Multivariate analysis V (pp. 501–522). Amsterdam: North Holland.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm.Journal of the Royal Statistical Society, Series B, 39, 1–38.
Gabriel, K. R., & Zamir, S. (1979). Lower rank approximation of matrices by least squares with any choice of weights.Technometrics, 21, 489–498.
Gifi, A. (1990).Nonlinear multivariate analysis. Chichester: Wiley.
Green, B. F. (1952). The orthogonal approximation of an oblique structure in factor analysis.Psychometrika, 17, 429–440.
Harman, H. H., & Jones, W. H. (1966). Factor analysis by minimizing residuals (Minres).Psychometrika, 31, 351–368.
Harshman, R. A. (1978, August).Models for analysis of asymmetrical relationships among N objects or stimuli. Paper presented at the First Joint Meeting of the Psychometric Society and the Society for Mathematical Psychology, Hamilton, Ontario.
Harshman, R. A., Green, P. E., Wind, Y., & Lundy, M. E. (1982). A model for the analysis of asymmetric data in marketing research,Marketing Science, 1, 205–242.
Harshman, R. A., & Lundy, M. E. (1984). The PARAFAC model for three-way factor analysis and multidimensional scaling. In H. G. Law, C. W. Snyder, J. A. Hattie, & R. P. McDonald (Eds.),Research methods for multimode data analysis (pp. 122–215). New York: Praeger.
Heiser, W. J. (1987). Correspondence Analysis with least absolute residuals.Computational Statistics and Data Analysis, 5, 337–356.
Heiser, W. J. (1995). Convergent computation by iterative majorization: theory and applications in multidimensional data analysis. In W. J. Krzanowski (Ed.),Recent advances in descriptive multivariate analysis (pp. 157–189). Oxford: Oxford University Press.
Jöreskog, K. G., & Sörbom, D. (1993).LISREL 8 User's guide. Chicago: Scientific Software International.
Kiers, H. A. L. (1989). An alternating least squares algorithm for fitting the two- and three-way DEDICOM model and the IDIOSCAL model.Psychometrika, 54, 515–521.
Kiers, H. A. L. (1990). Majorization as a tool for optimizing a class of matrix functions.Psychometrika, 55, 417–428.
Kiers, H. A. L. (1993). An alternating least squares algorithm for PARAFAC2 and DEDICOM3.Computational Statistics and Data Analysis, 16, 103–118.
Kiers, H. A. L., & ten Berge, J. M. F. (1992). Minimization of a class of matrix trace functions by means of refined majorization.Psychometrika, 57, 371–382.
Kiers, H. A. L., ten Berge, J. M. F., Takane, Y., & de Leeuw, J. (1990). A generalization of Takane's algorithm for DEDICOM.Psychometrika, 55, 151–158.
Takane, Y. (1985). Diagonal Estimation in DEDICOM.Proceedings of the 1985 Annual Meeting of the Behaviormetric Society (pp. 100–101). Sapporo, Japan: Behaviormetric Society.
ten Berge, J. M. F., & Kiers, H. A. L. (1989). Fitting the off-diagonal DEDICOM model in the least-squares sense by a generalization of the Harman & Jones MINRES procedure of factor analysis.Psychometrika, 54, 333–337.
ten Berge, J. M. F., & Kiers, H. A. L. (1993). An alternating least squares method for the weighted approximation of a symmetric matrix.Psychometrika, 58, 115–118.
ten Berge, J. M. F., Kiers, H. A. L., & Commandeur, J. J. F. (1993). Orthogonal Procrustes rotation for matrices with missing values.British Journal of Mathematical and Statistical Psychology, 46, 119–134.
Verboon, P. (1994).A robust approach to nonlinear multivariate analysis. Leiden: DSWO Press.
Verboon, P., & Heiser, W. J. (1992). Resistant orthogonal Procrustes analysis.Journal of Classification, 9, 237–256.
Verboon, P., & Heiser, W. J. (1994). Resistant lower rank approximation of matrices by iterative majorization.Computational Statistics and Data Analysis, 18, 457–467.
Author information
Authors and Affiliations
Additional information
This research has been made possible by a fellowship from the Royal Netherlands Academy of Arts and Sciences to the author.
Rights and permissions
About this article
Cite this article
Kiers, H.A.L. Weighted least squares fitting using ordinary least squares algorithms. Psychometrika 62, 251–266 (1997). https://doi.org/10.1007/BF02295279
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02295279