Skip to main content
Erschienen in: Journal of Classification 3/2021

01.04.2021

A Unified Theory of the Completeness of Q-Matrices for the DINA Model

verfasst von: Hans Friedrich Köhn, Chia-Yi Chiu

Erschienen in: Journal of Classification | Ausgabe 3/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Diagnostic classification models in educational measurement describe ability in a knowledge domain as a composite of specific binary skills called “cognitive attributes,” each of which an examinee may or may not have mastered. Attribute Hierarchy Models (AHMs) account for the possibility that attributes are dependent by imposing a hierarchical structure such that mastery of one or more attributes is a prerequisite of mastering one or more other attributes. Thus, the number of meaningfully defined attribute combinations is reduced, so that constructing a complete Q-matrix may be challenging. (The Q-matrix of a cognitively diagnostic test documents which attributes are required for solving which item; the Q-matrix is said to be complete if it guarantees the identifiability of all realizable proficiency classes among examinees.) For structured Q-matrices (i.e., the item attribute profiles are restricted to reflect the hierarchy postulated to underlie the attributes), the conditions of completeness have been established. However, sometimes, a structured Q-matrix cannot be assembled because the items of the test in question have attribute profiles that do not conform to the prerequisite structure imposed by the postulated attribute hierarchy. A Q-matrix composed of such items is called “unstructured.” In this article, the completeness conditions of unstructured Q-matrices for the DINA model are presented. Specifically, there exists an entire range of Q-matrices that are all complete for DINA-AHMs. Thus, the theoretical results presented here can be combined with extant insights about Q-completeness for models without attribute hierarchies into a unified framework on the completeness of Q-matrices for the DINA model.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Cai, Y., Tu, D., & Ding, S. (2018). Theorems and methods of a complete Q matrix with attribute hierarchies under restricted Q-matrix design. Frontiers in Psychology, 9, Article 1413. Cai, Y., Tu, D., & Ding, S. (2018). Theorems and methods of a complete Q matrix with attribute hierarchies under restricted Q-matrix design. Frontiers in Psychology, 9, Article 1413.
Zurück zum Zitat Chen, Y., Culpepper, S.A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of DINA Q matrix. Psychometrika, 83, 89–108.MathSciNetCrossRef Chen, Y., Culpepper, S.A., Chen, Y., & Douglas, J. (2018). Bayesian estimation of DINA Q matrix. Psychometrika, 83, 89–108.MathSciNetCrossRef
Zurück zum Zitat Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.MathSciNetCrossRef Chen, Y., Liu, J., Xu, G., & Ying, Z. (2015). Statistical analysis of Q-matrix based diagnostic classification models. Journal of the American Statistical Association, 110, 850–866.MathSciNetCrossRef
Zurück zum Zitat Chiu, C.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37, 598–618.CrossRef Chiu, C.-Y. (2013). Statistical refinement of the Q-matrix in cognitive diagnosis. Applied Psychological Measurement, 37, 598–618.CrossRef
Zurück zum Zitat Chiu, C.-Y., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.MathSciNetCrossRef Chiu, C.-Y., Douglas, J., & Li, X. (2009). Cluster analysis for cognitive diagnosis: Theory and applications. Psychometrika, 74, 633–665.MathSciNetCrossRef
Zurück zum Zitat Chiu, C.-Y., & Köhn, H.-F. (2015). Consistency of cluster analysis for cognitive diagnosis: The DINO model and the DINA model revisited. Applied Psychological Measurement, 39, 465–479.CrossRef Chiu, C.-Y., & Köhn, H.-F. (2015). Consistency of cluster analysis for cognitive diagnosis: The DINO model and the DINA model revisited. Applied Psychological Measurement, 39, 465–479.CrossRef
Zurück zum Zitat Chiu, C.-Y., & Köhn, H.-F. (2016). The Reduced RUM as a logit model:, Parameterization and constraints. Psychometrika, 81, 350–370.MathSciNetCrossRef Chiu, C.-Y., & Köhn, H.-F. (2016). The Reduced RUM as a logit model:, Parameterization and constraints. Psychometrika, 81, 350–370.MathSciNetCrossRef
Zurück zum Zitat Chung, M. (2014). Estimating the Q-matrix for Cognitive Diagnosis Models in a Bayesian Framework. Doctoral dissertation. Columbia University. Chung, M. (2014). Estimating the Q-matrix for Cognitive Diagnosis Models in a Bayesian Framework. Doctoral dissertation. Columbia University.
Zurück zum Zitat Chung, M. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 1–8.MathSciNetCrossRef Chung, M. (2019). A Gibbs sampling algorithm that estimates the Q-matrix for the DINA model. Journal of Mathematical Psychology, 93, 1–8.MathSciNetCrossRef
Zurück zum Zitat DeCarlo, L.T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468.CrossRef DeCarlo, L.T. (2012). Recognizing uncertainty in the Q-matrix via a Bayesian extension of the DINA model. Applied Psychological Measurement, 36, 447–468.CrossRef
Zurück zum Zitat de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.CrossRef de la Torre, J. (2008). An empirically based method of Q-matrix validation for the DINA model: Development and applications. Journal of Educational Measurement, 45, 343–362.CrossRef
Zurück zum Zitat de la Torre, J., & Chiu, C.-Y. (2016). A general method of empirical Q-matrix validation. Psychometrika, 81, 253–73.MathSciNetCrossRef de la Torre, J., & Chiu, C.-Y. (2016). A general method of empirical Q-matrix validation. Psychometrika, 81, 253–73.MathSciNetCrossRef
Zurück zum Zitat de la Torre, J., & Douglas, J.A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.MathSciNetCrossRef de la Torre, J., & Douglas, J.A. (2004). Higher-order latent trait models for cognitive diagnosis. Psychometrika, 69, 333–353.MathSciNetCrossRef
Zurück zum Zitat de la Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47, 227–249.CrossRef de la Torre, J., Hong, Y., & Deng, W. (2010). Factors affecting the item parameter estimation and classification accuracy of the DINA model. Journal of Educational Measurement, 47, 227–249.CrossRef
Zurück zum Zitat DiBello, L.V., Roussos, L.A., & Stout, W.F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 979–1030). Amsterdam: Elsevier. DiBello, L.V., Roussos, L.A., & Stout, W.F. (2007). Review of cognitively diagnostic assessment and a summary of psychometric models. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 979–1030). Amsterdam: Elsevier.
Zurück zum Zitat Fu, J., & Li, Y. (2007). An integrative review of cognitively diagnostic psychometric models. Chicago: Paper presented at the Annual Meeting of the National Council on Measurement in Education. Fu, J., & Li, Y. (2007). An integrative review of cognitively diagnostic psychometric models. Chicago: Paper presented at the Annual Meeting of the National Council on Measurement in Education.
Zurück zum Zitat Gierl, M. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44, 325–340.CrossRef Gierl, M. (2007). Making diagnostic inferences about cognitive attributes using the rule-space model and attribute hierarchy method. Journal of Educational Measurement, 44, 325–340.CrossRef
Zurück zum Zitat Gross, J., & George, A.C. (2014). On prerequisite relations between attributes in noncompensatory diagnostic classification. Methodology, 10(3), 100–107.CrossRef Gross, J., & George, A.C. (2014). On prerequisite relations between attributes in noncompensatory diagnostic classification. Methodology, 10(3), 100–107.CrossRef
Zurück zum Zitat Gu, Y., & Xu, G. (2020). Partial identifiability of restricted latent class models. Annals of Statistics, 48(4), 2082–2107.MathSciNetCrossRef Gu, Y., & Xu, G. (2020). Partial identifiability of restricted latent class models. Annals of Statistics, 48(4), 2082–2107.MathSciNetCrossRef
Zurück zum Zitat Gu, Y., & Xu, G. (2019a). The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika, 84, 468–483.MathSciNetCrossRef Gu, Y., & Xu, G. (2019a). The sufficient and necessary condition for the identifiability and estimability of the DINA model. Psychometrika, 84, 468–483.MathSciNetCrossRef
Zurück zum Zitat Haberman, S.J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 1031–1038). Amsterdam: Elsevier. Haberman, S.J., & von Davier, M. (2007). Some notes on models for cognitively based skill diagnosis. In Rao, C.R., & Sinharay, S. (Eds.) Handbook of Statistics: Vol.26. Psychometrics (pp. 1031–1038). Amsterdam: Elsevier.
Zurück zum Zitat Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.CrossRef Haertel, E.H. (1989). Using restricted latent class models to map the skill structure of achievement items. Journal of Educational Measurement, 26, 333–352.CrossRef
Zurück zum Zitat Haertel, E.H., & Wiley, D.E. (1994). Representation of ability structure: Implications for testing. In Fredriksen, N., Mislevy, R., & Bejar, I. (Eds.) Testing theory for a new generation of tests (pp. 359–384). Hillsdale: Erlbaum. Haertel, E.H., & Wiley, D.E. (1994). Representation of ability structure: Implications for testing. In Fredriksen, N., Mislevy, R., & Bejar, I. (Eds.) Testing theory for a new generation of tests (pp. 359–384). Hillsdale: Erlbaum.
Zurück zum Zitat Hartz, S.M. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No.3044108). Hartz, S.M. (2002). A Bayesian framework for the Unified Model for assessing cognitive abilities: Blending theory with practicality (Doctoral dissertation). Available from ProQuest Dissertations and Theses database. (UMI No.3044108).
Zurück zum Zitat Hartz, S.M., & Roussos, L.A. (2008). The Fusion Model for skill diagnosis: Blending theory with practicality. (Research report No. RR-08-71). Princeton: Educational Testing Service. Hartz, S.M., & Roussos, L.A. (2008). The Fusion Model for skill diagnosis: Blending theory with practicality. (Research report No. RR-08-71). Princeton: Educational Testing Service.
Zurück zum Zitat Henson, R.A., Templin, J.L., & Willse, J.T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.MathSciNetCrossRef Henson, R.A., Templin, J.L., & Willse, J.T. (2009). Defining a family of cognitive diagnosis models using log-linear models with latent variables. Psychometrika, 74, 191–210.MathSciNetCrossRef
Zurück zum Zitat Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.MathSciNetCrossRef Junker, B.W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.MathSciNetCrossRef
Zurück zum Zitat Köhn, H.-F., & Chiu, C.-Y. (2017). A procedure for assessing completeness of the Q-matrix of cognitive diagnosis models. Psychometrika, 82, 112–132.MathSciNetCrossRef Köhn, H.-F., & Chiu, C.-Y. (2017). A procedure for assessing completeness of the Q-matrix of cognitive diagnosis models. Psychometrika, 82, 112–132.MathSciNetCrossRef
Zurück zum Zitat Köhn, H.F., & Chiu, C.-Y. (2019). Attribute hierarchy models in cognitive diagnosis: identfiability of the latent attribute space and conditions for completeness of the Q-matrix. Journal of Classfication, 36, 541–565.CrossRef Köhn, H.F., & Chiu, C.-Y. (2019). Attribute hierarchy models in cognitive diagnosis: identfiability of the latent attribute space and conditions for completeness of the Q-matrix. Journal of Classfication, 36, 541–565.CrossRef
Zurück zum Zitat Leighton, J., & Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge: Cambridge University Press.CrossRef Leighton, J., & Gierl, M. (2007). Cognitive diagnostic assessment for education: Theory and applications. Cambridge: Cambridge University Press.CrossRef
Zurück zum Zitat Leighton, J.P., Gierl, M.J., & Hunka, S. (2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41, 205–236.CrossRef Leighton, J.P., Gierl, M.J., & Hunka, S. (2004). The attribute hierarchy model: An approach for integrating cognitive theory with assessment practice. Journal of Educational Measurement, 41, 205–236.CrossRef
Zurück zum Zitat Liu, C.-W., Andersson, B., & Skrondal, A. (2020). A constrained Metropolis-Hastings Robbins-Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85, 322–357.MathSciNetCrossRef Liu, C.-W., Andersson, B., & Skrondal, A. (2020). A constrained Metropolis-Hastings Robbins-Monro algorithm for Q matrix estimation in DINA models. Psychometrika, 85, 322–357.MathSciNetCrossRef
Zurück zum Zitat Liu, R., Huggins-Manley, A.C., & Bradshaw, L. (2016). The impact of q-matrix designs on diagnostic classification accuracy in the presence of attribute hierarchies. Educational and Psychological Measurement, 76, 220–240. Liu, R., Huggins-Manley, A.C., & Bradshaw, L. (2016). The impact of q-matrix designs on diagnostic classification accuracy in the presence of attribute hierarchies. Educational and Psychological Measurement, 76, 220–240.
Zurück zum Zitat Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36, 548–564.CrossRef Liu, J., Xu, G., & Ying, Z. (2012). Data-driven learning of Q-matrix. Applied Psychological Measurement, 36, 548–564.CrossRef
Zurück zum Zitat Liu, J., Xu, G., & Ying, Z. (2013). Theory of the self-learning Q-matrix. Bernoulli, 19, 1790–1817.MathSciNetMATH Liu, J., Xu, G., & Ying, Z. (2013). Theory of the self-learning Q-matrix. Bernoulli, 19, 1790–1817.MathSciNetMATH
Zurück zum Zitat Macready, G.B., & Dayton, C.M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 33, 379–416. Macready, G.B., & Dayton, C.M. (1977). The use of probabilistic models in the assessment of mastery. Journal of Educational Statistics, 33, 379–416.
Zurück zum Zitat Nichols, P.D., Chipman, S.E., & Brennan, R.L. (1995). Cognitively diagnostic assessment. Mahwah: Erlbaum. Nichols, P.D., Chipman, S.E., & Brennan, R.L. (1995). Cognitively diagnostic assessment. Mahwah: Erlbaum.
Zurück zum Zitat Rupp, A.A., & Templin, J.L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6, 219–262. Rupp, A.A., & Templin, J.L. (2008). Unique characteristics of diagnostic classification models: A comprehensive review of the current state-of-the-art. Measurement, 6, 219–262.
Zurück zum Zitat Rupp, A.A., Templin, J.L., & Henson, R.A. (2010). Diagnostic Measurement. Theory, Methods, and Applications. New York: Guilford. Rupp, A.A., Templin, J.L., & Henson, R.A. (2010). Diagnostic Measurement. Theory, Methods, and Applications. New York: Guilford.
Zurück zum Zitat Sessoms, J., & Henson, R.A. (2018). Applications of Diagnostic Classification Models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16(1), 1–17. Sessoms, J., & Henson, R.A. (2018). Applications of Diagnostic Classification Models: A literature review and critical commentary. Measurement: Interdisciplinary Research and Perspectives, 16(1), 1–17.
Zurück zum Zitat Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational and Behavioral Statistics, 12, 55–73.CrossRef Tatsuoka, K.K. (1985). A probabilistic model for diagnosing misconception in the pattern classification approach. Journal of Educational and Behavioral Statistics, 12, 55–73.CrossRef
Zurück zum Zitat Tatsuoka, K.K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. (Eds.) Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Erlbaum: Hillsdale. Tatsuoka, K.K. (1990). Toward an integration of item-response theory and cognitive error diagnosis. In Frederiksen, N., Glaser, R., Lesgold, A., & Shafto, M. (Eds.) Diagnostic monitoring of skill and knowledge acquisition (pp. 453–488). Erlbaum: Hillsdale.
Zurück zum Zitat Tatsuoka, K.K. (2009). Cognitive assessment. An introduction to the Rule Space Method. New York: Routledge/Taylor & Francis.CrossRef Tatsuoka, K.K. (2009). Cognitive assessment. An introduction to the Rule Space Method. New York: Routledge/Taylor & Francis.CrossRef
Zurück zum Zitat Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.MathSciNetCrossRef Templin, J., & Bradshaw, L. (2014). Hierarchical diagnostic classification models: A family of models for estimating and testing attribute hierarchies. Psychometrika, 79, 317–339.MathSciNetCrossRef
Zurück zum Zitat von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service. von Davier, M. (2005). A general diagnostic model applied to language testing data (Research report No. RR-05-16). Princeton, NJ: Educational Testing Service.
Zurück zum Zitat von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.MathSciNetCrossRef von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–301.MathSciNetCrossRef
Zurück zum Zitat Zhang, S. S., DeCarlo, L. T., & Ying, Z. (2013). Non-identifiability, equivalence classes, and attribute-specific classification in Q-matrix based cognitive diagnosis models. preprint, arXiv:1303.0426. Zhang, S. S., DeCarlo, L. T., & Ying, Z. (2013). Non-identifiability, equivalence classes, and attribute-specific classification in Q-matrix based cognitive diagnosis models. preprint, arXiv:1303.​0426.
Metadaten
Titel
A Unified Theory of the Completeness of Q-Matrices for the DINA Model
verfasst von
Hans Friedrich Köhn
Chia-Yi Chiu
Publikationsdatum
01.04.2021
Verlag
Springer US
Erschienen in
Journal of Classification / Ausgabe 3/2021
Print ISSN: 0176-4268
Elektronische ISSN: 1432-1343
DOI
https://doi.org/10.1007/s00357-021-09384-7

Weitere Artikel der Ausgabe 3/2021

Journal of Classification 3/2021 Zur Ausgabe

Premium Partner