Abstract
Standard procedures for drawing inferences from complex samples do not apply when the variable of interestϑ cannot be observed directly, but must be inferred from the values of secondary random variables that depend onϑ stochastically. Examples are proficiency variables in item response models and class memberships in latent class models. Rubin's “multiple imputation” techniques yield approximations of sample statistics that would have been obtained, hadϑ been observable, and associated variance estimates that account for uncertainty due to both the sampling of respondents and the latent nature ofϑ. The approach is illustrated with data from the National Assessment for Educational Progress.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersen, E. B., & Madsen, M. (1977). Estimating the parameters of a latent population distribution.Psychometrika, 42, 357–374.
Beaton, A. E. (1987).The NAEP 1983/84 technical report (NAEP Report 15-TR-20). Princeton: Educational Testing Service.
Bock, R. D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: An application of an EM algorithm.Psychometrika, 46, 443–459.
Bock, R. D., Mislevy, R. J., & Woodson, C. E. M. (1982). The next stage in educational assessment.Educational Researcher, 11, 4–11, 16.
Cassel, C. M., Särndal, C. E., & Wretman, J. H. (1977).Foundations of inference in survey sampling. New York: Wiley.
Cochran, W. G. (1977).Sampling techniques. New York: Wiley.
Dempster, A. P., Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm (with discussion).Journal of the Royal Statistical Society, Series B, 39, 1–38.
Ford, B. L. (1983). An overview of hot-deck procedures. In W. G. Madow, I. Olkin, & D. B. Rubin (Eds.),Incomplete data in sample surveys, Volume 2, Theory and bibliographies (pp. 185–207). New York: Academic Press.
Goldstein, H., & McDonald, R. P. (1988). A general model for the analysis of multilevel data.Psychometrika, 53, 455–467.
Johnson, E. G. (1987). Estimation of uncertainty due to sampling variation. In A. E. Beaton,The NAEP 1983/84 technical report (NAEP Report 15-TR-20, pp. 505–512). Princeton: Educational Testing Service.
Kelley, T. L. (1923).Statistical method. New York: Macmillan.
Kirsch, I. S., & Jungeblut, A. (1986).Literacy: Profile of America's young adults (Final Report, No. 16-PL-01). Princeton, NJ: National Assessment of Education Progress.
Laird, N. M. (1978). Nonparametric maximum likelihood estimation of a mixing distribution.Journal of the American Statistical Association, 73, 805–811.
Little, R. J. A., & Rubin, D. B. (1987).Statistical analysis with missing data. New York: Wiley.
Lord, F. M. (1965). A strong true-score theory, with applications.Psychometrika, 30, 239–270.
Lord, F. M. (1969). Estimating true score distributions in psychological testing (An empirical Bayes problem).Psychometrika, 34, 259–299.
Mislevy, R. J. (1984). Estimating latent distributions.Psychometrika, 49, 359–381.
Mislevy, R. J. (1985). Estimation of latent group effects.Journal of the American Statistical Association, 80, 993–997.
Mislevy, R. J., & Bock, R. D. (1983).BILOG: Item analysis and test scoring with binary logistic models [computer program]. Mooresville, IN: Scientific Software.
Mislevy, R. J., & Sheehan, K. M. (1987). Marginal estimation procedures. In A. E. Beaton,The NAEP 1983/84 technical report (NAEP Report 15-TR-20, pp. 293–360). Princeton: Educational Testing Service.
Muthén, B., & Satorra, A. (1989). Multilevel aspects of varying parameters in structural models. In R. D. Bock (Ed.),Multilevel analysis of educational data (pp. 87–99). San Diego: Academic Press.
National Assessment of Educational Progress (1985).The reading report card: Progress toward excellence in our schools. Princeton: Educational Testing Service.
Rigdon, S. E., & Tsutakawa, R. K. (1983). Parameter estimation in latent trait models.Psychometrika, 48, 567–574.
Rubin, D. B. (1977). Formalizing subjective notions about the effect of nonrespondents in sample surveys.Journal of the American Statistical Association, 72, 538–543.
Rubin, D. B. (1987).Multiple imputation for nonresponse in surveys. New York: Wiley.
Sanathanan, L., & Blumenthal, N. (1978). The logistic model and estimation of latent structure.Journal of the American Statistical Association, 73, 794–798.
Sheehan, K. M. (1985).M-GROUP: Estimation of group effects in multivariate models [computer program]. Princeton: Educational Testing Service.
Tanner, M., & Wong, W. (1987). The calculation of posterior distributions by data augmentation (with discussion).Journal of the American Statistical Association, 82, 528–550.
Zwick, R. (1987). Assessing the dimensionality of NAEP reading data.Journal of Educational Measurement, 24, 293–308.
Author information
Authors and Affiliations
Additional information
This research was supported by Grant No. NIE-G-83-0011 of the Office for Educational Research and Improvement, Center for Education Statistics, and Contract No. N00014-88-K-0304, R&T 4421552 from the Cognitive Sciences Program, Cognitive and Neural Sciences Division, Office of Naval Research. It does not necessarily reflect the views of either agency. I am grateful to R. Darrell Bock for calling my attention to the applicability of multiple imputation to the assessment setting; to Albert Beaton and Eugene Johnson for enlightening discussions on the topic; and to Henry Braun, Ben King, Debra Kline, Gary Phillips, Paul Rosenbaum, Don Rubin, John Tukey, Ming-Mei Wang, Kentaro Yamamoto, Rebecca Zwick, and two anonymous reviewers for comments on earlier drafts. Example 4 is based on the analysis of the 1984 National Assessment for Educational Progress reading survey, carried out at Educational Testing Service through the tireless efforts of too many people to mention by name, under the direction of Albert Beaton, Director of NAEP Data Analyses. David Freund, Bruce Kaplan, and Jennifer Nelson conducted additional analyses of the 1984 and 1988 data for the example.
Rights and permissions
About this article
Cite this article
Mislevy, R.J. Randomization-based inference about latent variables from complex samples. Psychometrika 56, 177–196 (1991). https://doi.org/10.1007/BF02294457
Received:
Revised:
Issue Date:
DOI: https://doi.org/10.1007/BF02294457