Skip to main content
SpringerLink
Log in

A Two-Tier Full-Information Item Factor Analysis Model with Applications

  • Published:
Psychometrika Aims and scope Submit manuscript

Abstract

Motivated by Gibbons et al.’s (Appl. Psychol. Meas. 31:4–19, 2007) full-information maximum marginal likelihood item bifactor analysis for polytomous data, and Rijmen, Vansteelandt, and De Boeck’s (Psychometrika 73:167–182, 2008) work on constructing computationally efficient estimation algorithms for latent variable models, a two-tier item factor analysis model is developed in this research. The modeling framework subsumes standard multidimensional IRT models, bifactor IRT models, and testlet response theory models as special cases. Features of the model lead to a reduction in the dimensionality of the latent variable space, and consequently significant computational savings. An EM algorithm for full-information maximum marginal likelihood estimation is developed. Simulations and real data demonstrations confirm the accuracy and efficiency of the proposed methods. Three real data sets from a large-scale educational assessment, a longitudinal public health survey, and a scale development study measuring patient reported quality of life outcomes are analyzed as illustrations of the model’s broad range of applicability.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Adams, R., & Wu, M. (2002). PISA 2000 technical report. Paris: Organization for Economic Cooperation and Development.

    Google Scholar 

  • Baker, F.B., & Kim, S.H. (2004). Item response theory: Parameter estimation techniques (2nd ed.). New York: Marcel Dekker.

    Google Scholar 

  • Bartholomew, D.J., & Tzamourani, P. (1999). The goodness-of-fit of latent trait models in attitude measurement. Sociological Methods and Research, 27, 525–546.

    Article  Google Scholar 

  • Bishop, Y.M.M., Fienberg, S.E., & Holland, P.W. (1975). Discrete multivariate analysis: theory and practice. Cambridge: MIT Press.

    Google Scholar 

  • Bock, R.D., & Aitkin, M. (1981). Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm. Psychometrika, 46, 443–459.

    Article  Google Scholar 

  • Bock, R.D., Gibbons, R., & Muraki, E. (1988). Full-information item factor analysis. Applied Psychological Measurement, 12, 261–280.

    Article  Google Scholar 

  • Bock, R.D., & Lieberman, M. (1970). Fitting a response model for n dichotomously scored items. Psychometrika, 35, 179–197.

    Article  Google Scholar 

  • Bollen, K.A. (1989). Structural equations with latent variables. New York: Wiley.

    Google Scholar 

  • Bolt, D. (2005). Limited and full information estimation of item response theory models. In A. Maydeu-Olivares & J.J. McArdle (Eds.), Contemporary psychometrics (pp. 27–71). Mahwah: Earlbaum.

    Google Scholar 

  • Bradlow, E.T., Wainer, H., & Wang, X. (1999). A Bayesian random effects model for testlets. Psychometrika, 64, 153–168.

    Article  Google Scholar 

  • Braeken, J., Tuerlinckx, F., & De Boeck, P. (2007). Copula functions for residual dependency. Psychometrika, 72, 393–411.

    Article  Google Scholar 

  • Cai, L. (2008a). A Metropolis-Hastings Robbins-Monro algorithm for maximum likelihood nonlinear latent structure analysis with a comprehensive measurement model. Unpublished doctoral dissertation, Department of Psychology, University of North Carolina.

  • Cai, L. (2008b). SEM of another flavour: two new applications of the supplemented EM algorithm. British Journal of Mathematical and Statistical Psychology, 61, 309–329.

    Article  PubMed  Google Scholar 

  • Cai, L. (2010). High-dimensional exploratory item factor analysis by a Metropolis-Hastings Robbins-Monro algorithm. Psychometrika, 7, 33–57.

    Article  Google Scholar 

  • Cai, L. (in press). Metropolis-Hastings Robbins-Monro algorithm for confirmatory item factor analysis. Journal of Educational and Behavioral Statistics.

  • Cai, L., du Toit, S.H.C., & Thissen, D. (in press). IRTPRO: Flexible, multidimensional, multiple categorical IRT modeling. Chicago: Scientific Software Internatonal, Inc. Computer software.

  • Cai, L., Maydeu-Olivares, A., Coffman, D.L., & Thissen, D. (2006). Limited-information goodness-of-fit testing of item response theory models for sparse 2p tables. British Journal of Mathematical and Statistical Psychology, 59, 173–194.

    Article  PubMed  Google Scholar 

  • Chen, W.H., & Thissen, D. (1997). Local dependence indices for item pairs using item response theory. Journal of Educational and Behavioral Statistics, 22, 265–289.

    Google Scholar 

  • Cudeck, R., Harring, J.R., & du Toit, S.H.C. (2009). Marginal maximum likelihood estimation of a latent variable model with interaction. Journal of Educational and Behavioral Statistics, 34, 131–144.

    Article  Google Scholar 

  • de Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.

    Google Scholar 

  • de la Torre, J., & Patz, R.J. (2005). Making the most of what we have: A practical application of multidimensional item response theory in test scoring. Journal of Educational and Behavioral Statistics, 30, 295–311.

    Article  Google Scholar 

  • Edwards, M.C. (in press). A Markov chain Monte Carlo approach to confirmatory item factor analysis. Psychometrika. doi:10.1007/S11336-010-9161-9.

  • Ennett, S.T., Foshee, V.A., Bauman, K.E., Hussong, A.M., Cai, L., Luz, H., et al. (2008). The social ecology of adolescent alcohol misuse. Child Development, 79, 1777–1791.

    Article  PubMed  Google Scholar 

  • Gibbons, R.D., Bock, R.D., Hedeker, D., Weiss, D.J., Segawa, E., Bhaumik, D.K., et al. (2007). Full-information item bifactor analysis of graded response data. Applied Psychological Measurement, 31, 4–19.

    Article  Google Scholar 

  • Gibbons, R.D., Grochocinski, V.J., Weiss, D.J., Bhaumik, D.K., Kupfer, D.J., Stover, A., et al. (2008). Using computerized adaptive testing to reduce the burden of mental health assessment. Psychiatric Services, 59, 361–368.

    Article  PubMed  Google Scholar 

  • Gibbons, R.D., & Hedeker, D. (1992). Full-information item bifactor analysis. Psychometrika, 57, 423–436.

    Article  Google Scholar 

  • Gibbons, R.D., & Hedeker, D. (2007). Bifactor. Chicago: Center for Health Statistics, University of Illinois at Chicago. Computer software.

    Google Scholar 

  • Glas, C.A.W., Wainer, H., & Bradlow, E.T. (2000). Maximum marginal likelihood and expected a posteriori estimation in testlet-based adaptive testing. In W.J. van der Linden, & C.A.W. Glas (Eds.), Computerized adaptive testing: Theory and practice (pp. 271–288). Boston: Kluwer Academic.

    Google Scholar 

  • Haberman, S.J. (1977). Log-linear models and frequency tables with small expected cell counts. The Annals of Statistics, 5, 1148–1169.

    Article  Google Scholar 

  • Hill, C.D. (2006). Two models for longitudinal item response data. Unpublished doctoral dissertation, Department of Psychology, University of North Carolina at Chapel Hill.

  • Jöreskog, K.G., & Sörbom, D. (2001). LISREL user’s guide. Chicago: Scientific Software Internatonal, Inc.

    Google Scholar 

  • Lehman, A.F. (1988). A quality of life interview for the chronically mentally ill. Evaluation and Program Planning, 11, 51–62.

    Article  Google Scholar 

  • Li, Y., Bolt, D.M., & Fu, J. (2006). A comparison of alternative models for testlets. Applied Psychological Measurement, 30, 3–21.

    Article  Google Scholar 

  • Lord, F.M., & Novick, M.R. (1968). Statistical theories of mental test scores. Reading: Addison-Wesley.

    Google Scholar 

  • Mardia, K.V., Kent, J.T., & Bibby, J.M. (1979). Multivariate analysis. London: Academic Press.

    Google Scholar 

  • Maydeu-Olivares, A., & Cai, L. (2006). A cautionary note on using G 2(dif) to assess relative model fit in categorical data analysis. Multivariate Behavioral Research, 41, 55–64.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Coffman, D.L. (2006). Random intercept item factor analysis. Psychological Methods, 11, 344–362.

    Article  PubMed  Google Scholar 

  • Maydeu-Olivares, A., & Joe, H. (2005). Limited and full information estimation and testing in 2n contingency tables: A unified framework. Journal of the American Statistical Association, 100, 1009–1020.

    Article  Google Scholar 

  • Maydeu-Olivares, A., & Joe, H. (2006). Limited information goodness-of-fit testing in multidimensional contingency tables. Psychometrika, 71, 713–732.

    Article  Google Scholar 

  • Moustaki, I. (2007). Factor analysis and latent structure of categorical and metric data. In R. Cudeck & R.C. Maccallum (Eds.), Factor analysis at 100: Historical developments and future directions. Mahwah: Laurence Erlbaum Associates.

    Google Scholar 

  • Muraki, E. (1992). A generalized partial credit model: Application of an EM algorithm. Applied Psychological Measurement, 16, 159–176.

    Article  Google Scholar 

  • Muraki, E., & Carlson, J.E. (1995). Full-information factor analysis for polytomous item responses. Applied Psychological Measurement, 19, 73–90.

    Article  Google Scholar 

  • Muthén, & Muthén (2008). Mplus (Version 5.0). Los Angeles: Author. Computer software.

  • Orlando, M., & Thissen, D. (2000). Likelihood-based Item-Fit indices for dichotomous item response theory models. Applied Psychological Measurement, 24, 50–64.

    Article  Google Scholar 

  • Reckase, M.D. (2009). Multidimentional item response theory. New York: Springer.

    Book  Google Scholar 

  • Reeve, B.B., Hays, R.D., Bjorner, J.B., Cook, K.F., Crane, P.K., Teresi, J.A., et al. (2007). Psychometric evaluation and calibration of health-related quality of life items banks: Plans for the patient-reported outcome measurement information system (PROMIS). Medical Care, 45, 22–31.

    Article  Google Scholar 

  • Reise, S.P., Morizot, J., & Hays, R.D. (2007). The role of the bifactor model in resolving dimensionality issues in health outcomes measures. Quality of Life Research, 16, 19–31.

    Article  PubMed  Google Scholar 

  • Rijmen, F. (2009a). Efficient full information maximum likelihood estimation for multidimensional IRT models (Tech. Rep. No. RR-09-03). Educational Testing Service.

  • Rijmen, F. (2009b). A hierarchical factor IRT model for items that are clustered at multiple levels. Paper presented at the 2009 international meeting of the Psychometric Society. Cambridge, UK.

  • Rijmen, F. (2009c). Three Multidimensional models for testlet-based tests: formal relations and an empirical comparison (Tech. Rep. No. RR-09-37). Educational Testing Service.

  • Rijmen, F. (in press-a). Formal relations and an empirical comparison between the bi-factor, the testlet, and a second-order multidimensional IRT model. Journal of Educational Measurement.

  • Rijmen, F. (in press-b). The use of graphs in latent variable modeling: Beyond visualization. In G.R. Hancock & G.B. Macready (Eds.), Advances in latent class analysis: A Festshrift in honor of C. Mitchell Dayton.

  • Rijmen, F., Vansteelandt, K., & De Boeck, P. (2008). Latent class models for diary method data: Parameter estimation by local computations. Psychometrika, 73, 167–182.

    Article  PubMed  Google Scholar 

  • Samejima, F. (1969). Estimation of latent ability using a response pattern of graded scores. Psychometric Monographs, 17.

  • Schilling, S., & Bock, R.D. (2005). High-dimensional maximum marginal likelihood item factor analysis by adaptive quadrature. Psychometrika, 70, 533–555.

    Google Scholar 

  • Stout, W. (1990). A new item response theory modeling approach with application to unidimensional assessment and ability estimation. Psychometrika, 55, 293–325.

    Article  Google Scholar 

  • te Marvelde, J., Glas, V.G.C., & van Damme, J. (2006). Application of multidimensional item response theory models to longitudinal data. Educational and Psychological Measurement, 66, 5–34.

    Article  Google Scholar 

  • Thissen, D. (1982). Marginal maximum likelihood estimation for the one-parameter logistic model. Psychometrika, 47, 175–186.

    Article  Google Scholar 

  • Thissen, D. (2003). MULTILOG 7 user’s guide. Chicago: Scientific Software Internatonal, Inc.

    Google Scholar 

  • Thissen, D., Cai, L., & Bock, R.D. (in press). The nominal categories item response model. In M. Nering & R. Ostini (Eds.), Handbook of polytomous item response theory models: developments and applications. New York: Taylor & Francis.

  • Thissen, D., & Steinberg, L. (2010). Using item response theory to disentangle constructs at different levels of generality. In S. Embretson (Ed.), Measuring psychological constructs: Advances in model-based approaches (pp. 123–144). Washington: American Psychological Association.

    Chapter  Google Scholar 

  • Thissen, D., Steinberg, L., & Mooney, J.A. (1989). Trace lines for testlets: A use of multiple-categorical-response models. Journal of Educational Measurement, 26, 247–260.

    Article  Google Scholar 

  • Thissen, D., & Wainer, H. (2001). Test scoring. Mahwah: Lawrence Erlbaum Associates.

    Google Scholar 

  • Tisak, J., & Meredith, W. (1989). Exploratory longitudinal factor analysis in multiple populations. Psychometrika, 54, 261–281.

    Article  Google Scholar 

  • Wainer, H., Bradlow, E.T., & Wang, X. (2007). Testlet response theory and its applications. New York: Cambridge University Press.

    Book  Google Scholar 

  • Wirth, R.J., & Edwards, M.C. (2007). Item factor analysis: Current approaches and future directions. Psychological Methods, 12, 58–79.

    Article  PubMed  Google Scholar 

  • Woods, C.M., & Thissen, D. (2006). Item response theory with estimation of the latent population distribution using spline-based densities. Psychometrika, 71, 281–301.

    Article  Google Scholar 

  • Yeatts, K., Stucky, B.D., Thissen, D., Irwin, D., Varni, J., DeWitt, E.M., et al. (2010). Construction of the Pediatric Asthma Impact Scale (PAIS) for the patient-reported outcome measurement information system (PROMIS). Journal of Asthma, 47, 295–302.

    Article  PubMed  Google Scholar 

  • Yung, Y.F., McLeod, L.D., & Thissen, D. (1999). On the relationship between the higher-order factor model and the hierarchical factor model. Psychometrika, 64, 113–128.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of California, Education and Psychology, Los Angeles, CA, 90095-1521, USA

    Li Cai

Authors
  1. Li Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Li Cai.

Additional information

Part of this research is made possible by a pre-doctoral advanced quantitative methodology training grant (R305B080016) from the Institute of Education Sciences, a statistical methodology grant from the Institute of Education Sciences (R305D100039), and a grant from the National Institute on Drug Abuse (R01DA026943). The author is enormously grateful to Drs. Darren DeWalt, Susan Ennett, Robert Gibbons, Anthony Lehman, and David Thissen for their permission to use the data sets for numerical illustrations. Data collection for the Context project was supported by a grant from the National Institute on Drug Abuse (R01DA13459). The development of the Pediatric Asthma Impact Scale was funded by National Institute of Arthritis And Musculoskeletal and Skin Diseases (1U01AR052181-01). The development of IRTPRO was supported by the National Cancer Institute in the form of an SBIR contract (#HHSN-2612007-00013C) awarded to Scientific Software International. The views expressed in this paper are the author’s alone and do not reflect the views and policies of the funding agencies or grantees.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Cai, L. A Two-Tier Full-Information Item Factor Analysis Model with Applications. Psychometrika 75, 581–612 (2010). https://doi.org/10.1007/s11336-010-9178-0

Download citation

  • Received:

  • Revised:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11336-010-9178-0

Keywords

Search

Navigation