Abstract
While item complexity is often considered as an item feature in test development, it is much less frequently attended to in the psychometric modeling of test items. Prior work suggests that item complexity may manifest through asymmetry in item characteristics curves (ICCs; Samejima in Psychometrika 65:319–335, 2000). In the current paper, we study the potential for asymmetric IRT models to inform empirically about underlying item complexity, and thus the potential value of asymmetric models as tools for item validation. Both simulation and real data studies are presented. Some psychometric consequences of ignoring asymmetry, as well as potential strategies for more effective estimation of asymmetry, are considered in discussion.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Baker, F. B., & Kim, S. H. (Eds.). (2004). Item response theory: Parameter estimation techniques. CRC Press.
Bazán, J. L., Branco, M. D., & Bolfarine, H. (2006). A skew item response model. Bayesian Analysis, 1(4), 861–892.
Bolfarine, H., & Bazán, J. L. (2010). Bayesian estimation of the logistic positive exponent IRT model. Journal of Educational and Behavioral Statistics, 35, 693–713.
Bolt, D. M., Deng, S., & Lee, S. (2014). IRT model misspecification and measurement of growth in vertical scaling. Journal of Educational Measurement, 51(2), 141–162.
Bolt, D. M., Kim, J.-S., Blanton, M., & Knuth, E. (2016). Applications of item response theory in mathematics education research. In A. Izsák, J. T. Remillard, & J. Templin (Eds.), Psychometric methods in mathematics education: Opportunities, challenges, and interdisciplinary collaborations. Journal for research in mathematics education monograph series. Reston, VA: National Council of Teachers of Mathematics.
Bolt, D. M., & Lall, V. F. (2003). Estimation of compensatory and noncompensatory multidimensional item response models using Markov chain Monte Carlo. Applied Psychological Measurement, 27(6), 395–414.
De Ayala, R. J. (2013). The theory and practice of item response theory. New York: Guilford Publications.
De Boeck, P., & Wilson, M. (2004). Explanatory item response models: A generalized linear and nonlinear approach. New York: Springer.
Embretson, S. (1984). A general latent trait model for response processes. Psychometrika, 49(2), 175–186.
Falk, C. F., & Cai, L. (2016). Semi-parametric item response functions in the context of guessing. Journal of Educational Measurement, 53, 229–247.
Hess, K. (2006). Exploring cognitive demand in instruction and assessment. Dover, NH: National Center for the Improvement of Educational Assessment. Retrieved from http://www.nciea.org/publications/DOK_ApplyingWebb_KH08.pdf.
Jannarone, R. J. (1986). Conjunctive item response theory kernels. Psychometrika, 51, 357–373.
Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25(3), 258–272.
Lee, S. (2015). A comparison of methods for recovery of asymmetric item characteristic curves in item response theory. Unpublished masters thesis. University of Wisconsin, Madison.
Maris, E. (1995). Psychometric latent response models. Psychometrika, 60(4), 523–547.
McDonald, R. P. (1967). Nonlinear factor analysis (Psychometric Monograph No. 15). Richmond, VA: Psychometric Corporation. Retrieved from http://www.psychometrika.org/journal/online/MN15.pdf.
Molenaar, D. (2014). Heteroscedastic latent trait models for dichotomous data. Psychometrika, 80(3), 625–644.
Molenaar, D., Dolan, C. V., & de Boeck, P. (2012). The heteroscedastic graded response model with a skewed latent trait: Testing statistical and substantive hypotheses related to skewed item category functions. Psychometrika, 77(3), 455–478.
Neidorf, T. S., Binkley, M., Gattis, K., & Nohara, D. (2006). Comparing mathematics content in the National Assessment of Educational Progress (NAEP), Trends in International Mathematics and Science Study (TIMSS), and Program for International Student Assessment (PISA) 2003 assessments. Technical Report, Institute of Education Sciences, NCES 2006-029.
Ramsay J. O. (2000). TESTGRAF: A computer program for nonparametric analysis of testing data. Unpublished manuscript, McGill University. ftp://ego.psych.mcgill.ca/pub/ramsay/testgraf
Reckase, M. D. (1985). The difficulty of test items that measure more than one ability. Applied Psychological Measurement, 9, 401–412.
Rizopoulos, D. (2006). ltm: An R package for latent variable modeling and item response theory analyses. Journal of Statistical Software, 17(5), 1–25.
Samejima, F. (1995). Acceleration model in the heterogeneous case of the general graded response model. Psychometrika, 60(4), 549–572.
Samejima, F. (2000). Logistic positive exponent family of models: Virtue of asymmetric item characteristic curves. Psychometrika, 65, 319–335.
San Martín, E., Del Pino, G., & De Boeck, P. (2006). IRT models for ability-based guessing. Applied Psychological Measurement, 30(3), 183–203.
Sympson, J. B. (1978). A model for testing with multidimensional items. In D. J. Weiss (Ed.), Proceedings of the 1977 Computerized Adaptive Testing Conference (pp. 82–98). Minneapolis: University of Minnesota, Department of Psychology, Psychometric Methods Program.
Tuerlinckx, F., & De Boeck, P. (2001). The effect of ignoring item interactions on the estimated discrimination parameters in item response theory. Psychological Methods, 6(2), 181–195.
van der Linden, W. J. (2007). A hierarchical framework for modeling speed and accuracy on test items. Psychometrika, 72(3), 287–308.
Webb, N. L. (2002). Depth-of-knowledge levels for four content areas. Madison: University of Wisconsin Center for Educational Research.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
About this article
Cite this article
Lee, S., Bolt, D.M. Asymmetric Item Characteristic Curves and Item Complexity: Insights from Simulation and Real Data Analyses. Psychometrika 83, 453–475 (2018). https://doi.org/10.1007/s11336-017-9586-5
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11336-017-9586-5