nach oben

Erschienen in:

2015 | OriginalPaper | Buchkapitel

Evaluation of Expert-Based Q-Matrices Predictive Quality in Matrix Factorization Models

verfasst von : Guillaume Durand, Nabil Belacel, Cyril Goutte

Erschienen in: Design for Teaching and Learning in a Networked World

Verlag: Springer International Publishing

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

Matrix factorization techniques are widely used to build collaborative filtering recommender systems. These recommenders aim at discovering latent variables or attributes that are supposed to explain and ultimately predict the interest of users. In cognitive modeling, skills and competencies are considered as key latent attributes to understand and assess student learning. For this purpose, Tatsuoka introduced the concept of Q-matrix to represent the mapping between skills and test items. In this paper we evaluate how predictive expert-created Q-matrices can be when used as a decomposition factor in a matrix factorization recommender. To this end, we developed an evaluation method using cross validation and the weighted least squares algorithm that measures the predictive accuracy of Q-matrices. Results show that expert-made Q-matrices can be reasonably accurate at predicting users success in specific circumstances that are discussed at the end of this paper.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Using Educational Analytics to Improve Test Performance

Nächstes Kapitel Moving Through MOOCS: Pedagogy, Learning Design and Patterns of Engagement

Weighted least squares method:http://en.wikipedia.org/wiki/Least_squares#Weighted_least_squares.

CDM: http://cran.r-project.org/web/packages/CDM/index.html.

Barnes, T.: The Q-matrix method: Mining student response data for knowledge. In: AAAI Educational Data Mining workshop. p. 39 (2005)

Beheshti, B., Desmarais, M.C.: Predictive performance of prevailing approaches to skills assessment techniques: Insights from real vs. synthetic data sets (2014)

Birenbaum, M., Kelly, A.E., Tatsuoka, K.K.: Diagnosing Knowledge States in Algebra Using the Rule Space Model. ETS research report, Educational Testing Service, Educational Testing Service Princeton, NJ (1992)

DeCarlo, L.T.: On the analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the Q-matrix. Appl. Psychol. Meas. 35(1), 8–26 (2011)CrossRef

Desmarais, M.C., Beheshti, B., Naceur, R.: Item to skills mapping: deriving a conjunctive Q-matrix from data. In: Cerri, S.A., Clancey, W.J., Papadourakis, G., Panourgia, K. (eds.) ITS 2012. LNCS, vol. 7315, pp. 454–463. Springer, Heidelberg (2012) CrossRef

Desmarais, M.C., Naceur, R.: A matrix factorization method for mapping items to skills and for enhancing expert-based Q-Matrices. In: Lane, H.C., Yacef, K., Mostow, J., Pavlik, P. (eds.) AIED 2013. LNCS, vol. 7926, pp. 441–450. Springer, Heidelberg (2013) CrossRef

Gierl, M.J., Leighton, J.P., Hunka, S.M.: Exploring the logic of Tatsuoka’s rule-space model for test development and analysis. Educ. Meas. Issues Pract. 19(3), 34–44 (2000). http://dx.doi.org/10.1111/j.1745-3992.2000.tb00036.xCrossRef

Junker, B.W., Sijtsma, K.: Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Appl. Psychol. Meas. 25(3), 258–272 (2001). http://apm.sagepub.com/content/25/3/258.abstractMathSciNetCrossRef

Koedinger, K.R., Baker, R.S.J.D., Cunningham, K., Skogsholm, A., Leber, B., Stamper, J., Ventura, S., Pechenizkiy, M., Baker, R.S.J.D.: A data repository for the EDM community: The PSLC DataShop. In: Romero, C. (ed.) Handbook of Educational Data Mining, pp. 43–56. CRC Press, Boca Raton (2010)

10.

Koren, Y., Bell, R., Volinsky, C.: Matrix factorization techniques for recommender systems. Computer 42(8), 30–37 (2009). http://dx.doi.org/10.1109/MC.2009.263CrossRef

11.

Lan, A., Studer, C., Baraniuk, R.: Quantized matrix completion for personalized learning. In: 7th International Conference on Educational Data Mining, pp. 280–283 (2014)

12.

Liu, J., Xu, G., Ying, Z.: Data-driven learning of Q-matrix. Appl. Psychol. Meas. 36(7), 548–564 (2012)CrossRef

13.

Mislevy, R.J.: Test theory reconceived. J. Educ. Meas. 33(4), 379–416 (1996). http://dx.doi.org/10.1111/j.1745-3984.1996.tb00498.xCrossRef

14.

Nelder, J.A., Mead, R.: A simplex method for functional minimization. Comput. J. 7, 308–313 (1965)MathSciNetCrossRefMATH

15.

Ritter, S., Anderson, J., Koedinger, K., Corbett, A.: Cognitive tutor: applied research in mathematics education. Psychon. Bull. Rev. 14(2), 249–255 (2007). http://dx.doi.org/10.3758/BF03194060CrossRef

16.

Rupp, A.A., Templin, J.: The effects of q-matrix misspecification on parameter estimates and classification accuracy in the dina model. Educ. Psychol. Meas. 68(1), 78–96 (2008). http://epm.sagepub.com/content/68/1/78.abstractMathSciNetCrossRef

17.

Stamper, J., Ritter, S.: Cog model discovery experiment fall 2011. dataset 605 in datashop (2013). https://pslcdatashop.web.cmu.edu/datasetinfo?datasetid=605. Accessed

18.

Stamper, J.C., Koedinger, K.R.: Human-machine student model discovery and improvement using DataShop. In: Biswas, G., Bull, S., Kay, J., Mitrovic, A. (eds.) AIED 2011. LNCS, vol. 6738, pp. 353–360. Springer, Heidelberg (2011) CrossRef

19.

Su, Y.L., Choi, K.M., Lee, W.C., Choi, T., McAninch, M.: Hierarchical cognitive diagnostic analysis for TIMSS] 2003 mathematics. Technical report 35, Center for Advanced Studies in Measurement and Assessment (CASMA), University of Iowa (2013)

20.

Sun, Y., Ye, S., Inoue, S., Sun, Y.: Alternating recursive method for Q-matrix learning. In: 7th International Conference on Educational Data Mining, pp. 14–20 (2014)

21.

Tatsuoka, K.K.: Rule space: an approach for dealing with misconceptions based on item response theory. J. Educ. Meas. 20(4), 345–354 (1983)CrossRef

22.

Tatsuoka, K.K.: Analysis of errors in fraction addition and subtraction problems. Final report, Computer-based Education Research Laboratory, University of Illinois at Urbana-Champaign (1984)

23.

Tatsuoka, K.K.: Toward an integration of item-response theory and cognitive error diagnosis. In: Frederiksen, N., et al. (eds.) Diagnostic monitoring of skill and knowledge acquisition, pp. 453–488. Lawrence Erlbaum Associates, Hillsdale, NJ (1990)

24.

Tatsuoka, K.K.: Item Construction and Psychometric Models Appropriate for Constructed Responses. ETS research report, Educational Testing Service, Educational Testing Service Princeton, NJ (1991)

25.

Tatsuoka, K.K.: Architecture of knowledge structures and cognitive diagnosis: a statistical pattern recognition and classification approach. In: Nichols, P.D., Chipman, S.F., Brennan, R. (eds.) Cognitively Diagnostic Assessment, pp. 327–359. Lawrence Erlbaum Associates, Hillsdale (1995)

26.

The MathWorks, I.: Matlab 7, function reference. In: Matlab 7. Natick, Massachusetts (2008)

27.

de la Torre, J.: An empirically based method of q-matrix validation for the dina model: development and applications. J. Educ. Meas. 45(4), 343–362 (2008). http://dx.doi.org/10.1111/j.1745-3984.2008.00069.xCrossRef

28.

de la Torre, J.: DINA model and parameter estimation: a didactic. J. Educ. Behav. Stat. 34(1), 115–130 (2009)CrossRef

29.

de la Torre, J., Douglas, J.: Model evaluation and multiple strategies in cognitive diagnosis: an analysis of fraction subtraction data. Psychometrika 73(4), 595–624 (2008)MathSciNetCrossRefMATH

30.

de la Torre, J., Douglas, J.A.: Higher-order latent trait models for cognitive diagnosis. Psychometrika 69(3), 333–353 (2004)MathSciNetCrossRefMATH

Titel: Evaluation of Expert-Based Q-Matrices Predictive Quality in Matrix Factorization Models
verfasst von: Guillaume Durand
Nabil Belacel
Cyril Goutte
Verlag: Springer International Publishing
Buch: Design for Teaching and Learning in a Networked World
Print ISBN: 978-3-319-24257-6

Electronic ISBN: 978-3-319-24258-3

Copyright-Jahr: 2015
DOI: https://doi.org/10.1007/978-3-319-24258-3_5

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner