Skip to main content

2017 | OriginalPaper | Buchkapitel

4. A Review of Recent Advances in Adaptive Assessment

verfasst von : Jill-Jênn Vie, Fabrice Popineau, Éric Bruillard, Yolaine Bourda

Erschienen in: Learning Analytics: Fundaments, Applications, and Trends

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Computerized assessments are an increasingly popular way to evaluate students. They need to be optimized so that students can receive an accurate evaluation in as little time as possible. Such optimization is possible through learning analytics and computerized adaptive tests (CATs): the next question is then chosen according to the previous responses of the student, thereby making assessment more efficient. Using the data collected from previous students in non-adaptive tests, it is thus possible to provide formative adaptive tests to new students by telling them what to do next. This chapter reviews several models of CATs found in various fields, together with their main characteristics. We then compare these models empirically on real data. We conclude with a discussion of future research directions for computerized assessments.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
Zurück zum Zitat Altiner C (2011) Integrating a computer-based flashcard program into academic vocabulary learning. Master’s thesis, Iowa State University Altiner C (2011) Integrating a computer-based flashcard program into academic vocabulary learning. Master’s thesis, Iowa State University
Zurück zum Zitat Baker RS, Inventado PS (2014) Educational data mining and learning analytics. In: Learning analytics. Springer, New York, pp 61–75 Baker RS, Inventado PS (2014) Educational data mining and learning analytics. In: Learning analytics. Springer, New York, pp 61–75
Zurück zum Zitat Barnes T (2005) The q-matrix method: mining student response data for knowledge. In: American Association for Artificial Intelligence 2005 educational data mining workshop Barnes T (2005) The q-matrix method: mining student response data for knowledge. In: American Association for Artificial Intelligence 2005 educational data mining workshop
Zurück zum Zitat Bergner Y, Droschler S, Kortemeyer G, Rayyan S, Seaton D, Pritchard DE (2012) Model-based collaborative filtering analysis of student response data: machinelearning item response theory. International Educational Data Mining Society Bergner Y, Droschler S, Kortemeyer G, Rayyan S, Seaton D, Pritchard DE (2012) Model-based collaborative filtering analysis of student response data: machinelearning item response theory. International Educational Data Mining Society
Zurück zum Zitat Chang HH (2015) Psychometrics behind computerized adaptive testing. Psychometrika 80(1):1–20 Chang HH (2015) Psychometrics behind computerized adaptive testing. Psychometrika 80(1):1–20
Zurück zum Zitat Chatti MA, Dyckhoff AL, Schroeder U, Thüs H (2012) A reference model for learning analytics. Int J Technol Enhanced Learning 4(5–6):318–331 Chatti MA, Dyckhoff AL, Schroeder U, Thüs H (2012) A reference model for learning analytics. Int J Technol Enhanced Learning 4(5–6):318–331
Zurück zum Zitat Chen S, Choi A, Darwiche A (2015) Computer adaptive testing using the same-decision probability. In: 12th Annual Bayesian modeling applications workshop (BMAW) Chen S, Choi A, Darwiche A (2015) Computer adaptive testing using the same-decision probability. In: 12th Annual Bayesian modeling applications workshop (BMAW)
Zurück zum Zitat Clement B, Roy D, Oudeyer PY, Lopes M (2015) Multi-armed bandits for intelligent tutoring systems. JEDM-J Educ Data Mining 7(2):20–48 Clement B, Roy D, Oudeyer PY, Lopes M (2015) Multi-armed bandits for intelligent tutoring systems. JEDM-J Educ Data Mining 7(2):20–48
Zurück zum Zitat Davier M (2005) A general diagnostic model applied to language testing data. ETS research report series 2:i–35 Davier M (2005) A general diagnostic model applied to language testing data. ETS research report series 2:i–35
Zurück zum Zitat DeCarlo LT (2010) On the analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the q-matrix. Appl Psychol Measure 35(1):8–26 DeCarlo LT (2010) On the analysis of fraction subtraction data: the DINA model, classification, latent class sizes, and the q-matrix. Appl Psychol Measure 35(1):8–26
Zurück zum Zitat Desmarais MC, Baker RS (2012) A review of recent advances in learner and skill modeling in intelligent learning environments. User Model User-Adap Inter 22(1–2):9–38CrossRef Desmarais MC, Baker RS (2012) A review of recent advances in learner and skill modeling in intelligent learning environments. User Model User-Adap Inter 22(1–2):9–38CrossRef
Zurück zum Zitat Desmarais MC et al (2011) Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: 4th international conference on educational data mining, EDM, pp 41–50 Desmarais MC et al (2011) Conditions for effectively deriving a q-matrix from data with non-negative matrix factorization. In: 4th international conference on educational data mining, EDM, pp 41–50
Zurück zum Zitat Doignon JP, Falmagne JC (2012) Knowledge spaces. Springer Science & Business Media, Berlin Doignon JP, Falmagne JC (2012) Knowledge spaces. Springer Science & Business Media, Berlin
Zurück zum Zitat Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT (2013) Improving students learning with effective learning techniques promising directions from cognitive and educational psychology. Psychol Sci Public Interest 14(1):4–58CrossRef Dunlosky J, Rawson KA, Marsh EJ, Nathan MJ, Willingham DT (2013) Improving students learning with effective learning techniques promising directions from cognitive and educational psychology. Psychol Sci Public Interest 14(1):4–58CrossRef
Zurück zum Zitat Executive Office of the President, Podesta J (2014) Big data: seizing opportunities, preserving values. Technical report, The White House Executive Office of the President, Podesta J (2014) Big data: seizing opportunities, preserving values. Technical report, The White House
Zurück zum Zitat Falmagne JC, Cosyn E, Doignon JP, Thiéry N (2006) The assessment of knowledge, in theory and in practice. In: Formal concept analysis. Springer, Heidelberg, pp 61–79 Falmagne JC, Cosyn E, Doignon JP, Thiéry N (2006) The assessment of knowledge, in theory and in practice. In: Formal concept analysis. Springer, Heidelberg, pp 61–79
Zurück zum Zitat Ferguson R (2012) Learning analytics: drivers, developments and challenges. Int J Technol Enhanced Learning 4(5–6):304–317CrossRef Ferguson R (2012) Learning analytics: drivers, developments and challenges. Int J Technol Enhanced Learning 4(5–6):304–317CrossRef
Zurück zum Zitat Goggins SP, Xing W, Chen X, Chen B, Wadholm B (2015) Learning analytics at “small” scale: exploring a complexity-grounded model for assessment automation. J UCS 21(1):66–92 Goggins SP, Xing W, Chen X, Chen B, Wadholm B (2015) Learning analytics at “small” scale: exploring a complexity-grounded model for assessment automation. J UCS 21(1):66–92
Zurück zum Zitat Golbandi N, Koren Y, Lempel R (2011) Adaptive bootstrapping of recommender systems using decision trees. In: Proceedings of the fourth ACM international conference on Web search and data mining, ACM, pp 595–604 Golbandi N, Koren Y, Lempel R (2011) Adaptive bootstrapping of recommender systems using decision trees. In: Proceedings of the fourth ACM international conference on Web search and data mining, ACM, pp 595–604
Zurück zum Zitat Hambleton RK, Swaminathan H (1985) Item response theory: principles and applications, vol 7. Springer Science & Business Media, New York Hambleton RK, Swaminathan H (1985) Item response theory: principles and applications, vol 7. Springer Science & Business Media, New York
Zurück zum Zitat Han KT (2013) Item pocket method to allow response review and change in computerized adaptive testing. Appl Psychol Meas 37(4):259–275MathSciNetCrossRef Han KT (2013) Item pocket method to allow response review and change in computerized adaptive testing. Appl Psychol Meas 37(4):259–275MathSciNetCrossRef
Zurück zum Zitat Huebner A (2010) An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract Assess Res Eval 15(3):7 Huebner A (2010) An overview of recent developments in cognitive diagnostic computer adaptive assessments. Pract Assess Res Eval 15(3):7
Zurück zum Zitat Kickmeier-Rust MD, Albert D (2015) Competence-based knowledge space theory. In: Measuring and visualizing learning in the information-rich classroom. Routledge, New-York and London, p 109 Kickmeier-Rust MD, Albert D (2015) Competence-based knowledge space theory. In: Measuring and visualizing learning in the information-rich classroom. Routledge, New-York and London, p 109
Zurück zum Zitat Koedinger KR, McLaughlin EA, Stamper JC (2012) Automated student model improvement. International Educational Data Mining Society Koedinger KR, McLaughlin EA, Stamper JC (2012) Automated student model improvement. International Educational Data Mining Society
Zurück zum Zitat Lan AS, Waters AE, Studer C, Baraniuk RG (2014) Sparse factor analysis for learning and content analytics. J Mach Learn Res 15(1):1959–2008MathSciNetMATH Lan AS, Waters AE, Studer C, Baraniuk RG (2014) Sparse factor analysis for learning and content analytics. J Mach Learn Res 15(1):1959–2008MathSciNetMATH
Zurück zum Zitat Leighton JP, Gierl MJ, Hunka SM (2004) The attribute hierarchy method for cognitive assessment: a variation on tatsuoka’s rule-space approach. J Educ Meas 41(3):205–237CrossRef Leighton JP, Gierl MJ, Hunka SM (2004) The attribute hierarchy method for cognitive assessment: a variation on tatsuoka’s rule-space approach. J Educ Meas 41(3):205–237CrossRef
Zurück zum Zitat Mandin S, Guin N (2014) Basing learner modelling on an ontology of knowledge and skills. In: 2014 IEEE 14th international conference on advanced learning technologies (ICALT). IEEE, pp 321–323 Mandin S, Guin N (2014) Basing learner modelling on an ontology of knowledge and skills. In: 2014 IEEE 14th international conference on advanced learning technologies (ICALT). IEEE, pp 321–323
Zurück zum Zitat Manouselis N, Drachsler H, Vuorikari R, Hummel H, Koper R (2011) Recommender systems in technology enhanced learning. In: Recommender systems handbook, Springer Science & Business Media, New-York, pp 387–415 Manouselis N, Drachsler H, Vuorikari R, Hummel H, Koper R (2011) Recommender systems in technology enhanced learning. In: Recommender systems handbook, Springer Science & Business Media, New-York, pp 387–415
Zurück zum Zitat Mislevy RJ, Behrens JT, Dicerbo KE, Levy R (2012) Design and discovery in educational assessment: evidence-centered design, psychometrics, and educational data mining. JEDM-J Educ Data Mining 4(1):11–48 Mislevy RJ, Behrens JT, Dicerbo KE, Levy R (2012) Design and discovery in educational assessment: evidence-centered design, psychometrics, and educational data mining. JEDM-J Educ Data Mining 4(1):11–48
Zurück zum Zitat Peña-Ayala A (2014) Educational data mining: a survey and a data mining-based analysis of recent works. Expert Syst Appl 41(4):1432–1462 Peña-Ayala A (2014) Educational data mining: a survey and a data mining-based analysis of recent works. Expert Syst Appl 41(4):1432–1462
Zurück zum Zitat Reckase M (2009) Multidimensional item response theory, vol 150. Springer, New York Reckase M (2009) Multidimensional item response theory, vol 150. Springer, New York
Zurück zum Zitat Redecker C, Johannessen O (2013) Changing assessment towards a new assessment paradigm using ICT. Eur J Educ 48(1):79–96 Redecker C, Johannessen O (2013) Changing assessment towards a new assessment paradigm using ICT. Eur J Educ 48(1):79–96
Zurück zum Zitat Rupp A, Levy R, Dicerbo KE, Sweet SJ, Crawford AV, Calico T, Benson M, Fay D, Kunze KL, Mislevy RJ et al (2012) Putting ecd into practice: the interplay of theory and data in evidence models within a digital learning environment. JEDM—J Educ Data Mining 4(1):49–110 Rupp A, Levy R, Dicerbo KE, Sweet SJ, Crawford AV, Calico T, Benson M, Fay D, Kunze KL, Mislevy RJ et al (2012) Putting ecd into practice: the interplay of theory and data in evidence models within a digital learning environment. JEDM—J Educ Data Mining 4(1):49–110
Zurück zum Zitat Shute VJ, Ventura M, Kim YJ (2013) Assessment and learning of qualitative physics in Newton’s playground. J Educ Res 106(6):423–430CrossRef Shute VJ, Ventura M, Kim YJ (2013) Assessment and learning of qualitative physics in Newton’s playground. J Educ Res 106(6):423–430CrossRef
Zurück zum Zitat Shute VJ, Leighton JP, Jang EE, Chu MW (2016) Advances in the science of assessment. Educ Assess 21(1):34–59CrossRef Shute VJ, Leighton JP, Jang EE, Chu MW (2016) Advances in the science of assessment. Educ Assess 21(1):34–59CrossRef
Zurück zum Zitat Su YL, Choi K, Lee W, Choi T, McAninch M (2013) Hierarchical cognitive diagnostic analysis for TIMSS 2003 mathematics. Centre Adv Stud Meas Assess 35:1–71 Su YL, Choi K, Lee W, Choi T, McAninch M (2013) Hierarchical cognitive diagnostic analysis for TIMSS 2003 mathematics. Centre Adv Stud Meas Assess 35:1–71
Zurück zum Zitat Tempelaar DT, Rienties B, Giesbers B (2015) In search for the most informative data for feedback generation: learning analytics in a data-rich context. Comput Hum Behav 47:157–167CrossRef Tempelaar DT, Rienties B, Giesbers B (2015) In search for the most informative data for feedback generation: learning analytics in a data-rich context. Comput Hum Behav 47:157–167CrossRef
Zurück zum Zitat Thai-Nghe N, Drumond L, Horváth T, Schmidt-Thieme L et al (2011) Multi-relational factorization models for predicting student performance. In: Proceedings of the KDD workshop on knowledge discovery in educational data, Citeseer Thai-Nghe N, Drumond L, Horváth T, Schmidt-Thieme L et al (2011) Multi-relational factorization models for predicting student performance. In: Proceedings of the KDD workshop on knowledge discovery in educational data, Citeseer
Zurück zum Zitat Toscher A, Jahrer M (2010) Collaborative filtering applied to educational data mining. KDD Cup 2010 Toscher A, Jahrer M (2010) Collaborative filtering applied to educational data mining. KDD Cup 2010
Zurück zum Zitat Ueno M, Songmuang P (2010) Computerized adaptive testing based on decision tree. In: 2010 10th IEEE international conference on advanced learning technologies. IEEE, pp 191–193 Ueno M, Songmuang P (2010) Computerized adaptive testing based on decision tree. In: 2010 10th IEEE international conference on advanced learning technologies. IEEE, pp 191–193
Zurück zum Zitat Verbert K, Drachsler H, Manouselis N, Wolpers M, Vuorikari R, Duval E (2011) Dataset-driven research for improving recommender systems for learning. In: Proceedings of the 1st international conference on learning analytics and knowledge. ACM, pp 44–53 Verbert K, Drachsler H, Manouselis N, Wolpers M, Vuorikari R, Duval E (2011) Dataset-driven research for improving recommender systems for learning. In: Proceedings of the 1st international conference on learning analytics and knowledge. ACM, pp 44–53
Zurück zum Zitat Verhelst ND (2012) Profile analysis: a closer look at the PISA 2000 reading data. Scandinavi J Educ Res 56(3):315–332CrossRef Verhelst ND (2012) Profile analysis: a closer look at the PISA 2000 reading data. Scandinavi J Educ Res 56(3):315–332CrossRef
Zurück zum Zitat Vie JJ, Popineau F, Bourda Y, Bruillard (2016) Adaptive testing with a general diagnostic model. In: Design for teaching and learning in a networked world: 11th European conference on technology enhanced learning, EC-TEL 2016, Lyon, France, September 12–16, Proceedings (Springer, to appear) Vie JJ, Popineau F, Bourda Y, Bruillard (2016) Adaptive testing with a general diagnostic model. In: Design for teaching and learning in a networked world: 11th European conference on technology enhanced learning, EC-TEL 2016, Lyon, France, September 12–16, Proceedings (Springer, to appear)
Zurück zum Zitat Vygotsky LS (1980) Mind in society: the development of higher psychological processes. Harvard university press, Cambridge Vygotsky LS (1980) Mind in society: the development of higher psychological processes. Harvard university press, Cambridge
Zurück zum Zitat Wang S, Fellouris G, Chang HH (2015) Sequential design for computerized adaptive testing that allows for response revision. arXiv preprint arXiv:150101366 Wang S, Fellouris G, Chang HH (2015) Sequential design for computerized adaptive testing that allows for response revision. arXiv preprint arXiv:​150101366
Zurück zum Zitat Wang S, Lin H, Chang HH, Douglas J (2016) Hybrid computerized adaptive testing: from group sequential design to fully sequential design. J Educ Meas 53(1):45–62CrossRef Wang S, Lin H, Chang HH, Douglas J (2016) Hybrid computerized adaptive testing: from group sequential design to fully sequential design. J Educ Meas 53(1):45–62CrossRef
Zurück zum Zitat Winters T, Shelton C, Payne T, Mei G (2005) Topic extraction from item-level grades. In: American Association for Artificial Intelligence 2005 workshop on educational datamining, Pittsburgh, PA, vol 1, p 3 Winters T, Shelton C, Payne T, Mei G (2005) Topic extraction from item-level grades. In: American Association for Artificial Intelligence 2005 workshop on educational datamining, Pittsburgh, PA, vol 1, p 3
Zurück zum Zitat Xu X, Chang H, Douglas J (2003) A simulation study to compare CAT strategies for cognitive diagnosis. In: Annual meeting of the American Educational Research Association, Chicago Xu X, Chang H, Douglas J (2003) A simulation study to compare CAT strategies for cognitive diagnosis. In: Annual meeting of the American Educational Research Association, Chicago
Zurück zum Zitat Yan D, von Davier AA, Lewis C (2014) Computerized multistage testing. CRC Press, Boca Raton Yan D, von Davier AA, Lewis C (2014) Computerized multistage testing. CRC Press, Boca Raton
Zurück zum Zitat Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graphical Stat 15(2):265–286MathSciNetCrossRef Zou H, Hastie T, Tibshirani R (2006) Sparse principal component analysis. J Comput Graphical Stat 15(2):265–286MathSciNetCrossRef
Metadaten
Titel
A Review of Recent Advances in Adaptive Assessment
verfasst von
Jill-Jênn Vie
Fabrice Popineau
Éric Bruillard
Yolaine Bourda
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-52977-6_4