Skip to main content
Fachgebiete
Automobil + Motoren Bauwesen + Immobilien Business IT + Informatik Elektrotechnik + Elektronik Energie + Nachhaltigkeit Finance + Banking Management + Führung Marketing + Vertrieb Maschinenbau + Werkstoffe Versicherung + Risiko
DE
EN
Bücher
Zeitschriften
Themenseiten
Organisationspsychologie Projektmanagement Marketing Smart Manufacturing
Weitere Formate
Podcasts Webinare Technik Webinare Wirtschaft Kongresse Veranstaltungskalender Awards
Jetzt Einzelzugang starten
Zugang für Unternehmen
Referenzkunden
SLX-Digitalkonferenz: Zukunftswerkstatt 2024

Mehr Umsatz mit Top-Kontakten

Am 7. Mai erfahren Sie, wie Vertriebsteams Leadgenerierung, Leadnurturing und Leadstrategien effizient steuern können.
Erweiterte Suche
Anmelden
nach oben
Journal of Computers in Education
Erschienen in: Journal of Computers in Education 2/2021

10.01.2021

Development and validation of online cognitive diagnostic assessment with ordered multiple-choice items for ‘Multiplication of Time’

verfasst von: Huan Chin, Cheng Meng Chew, Hooi Lian Lim

Erschienen in: Journal of Computers in Education | Ausgabe 2/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper describes the process of developing and validating an online cognitive diagnostic assessment (CDA) with ordered multiple-choice (OMC) items for assessing pupils’ mastery of attributes related to ‘Multiplication of Time’ and reports the empirical evidence regarding item quality, validity of the cognitive models and the reliability of the online CDA with OMC items. The sample of the study consisted of 90 Grade Four students in Penang state, Malaysia. The data were analysed using two measurement models, namely classical test theory and attribute hierarchy method. The findings of the study indicated that the instrument developed consisted of good quality OMC items with robust psychometric properties. The linear hierarchical structure of the three cognitive models was supported by moderate correlations among the attributes in each cognitive model as well as the decrease of mean attribute mastery probability with the increase of attribute complexity. With the satisfactory model-data fit, the inferences made about pupils’ attribute mastery based on their performance in the online CDA with OMC items were valid. The online CDA with OMC items developed in this study was found to be reliable at both the attribute and assessment levels. The online CDA with OMC items could be used in the mathematics classrooms for supporting teachers in diagnosing pupils’ cognitive strengths and weaknesses for ‘Multiplication of Time’.
Vorheriger Artikel Flipping an EFL classroom with the LINE application: students’ performance and perceptions

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Jetzt informieren
Literatur
Almond, R., Steinberg, L., & Mislevy, R. (2002). Enhancing the design and delivery of assessment systems: A four-process architecture. The Journal of Technology Learning and Assessment, 1(5), 1–65.
Alonzo, A. C., & Steedle, J. T. (2009). Developing and assessing a force and motion learning progression. Science Education, 93(3), 389–421.
Alves, C. B. (2012). Making diagnostic inferences about student performance on the Alberta Education Diagnostic Mathematics Project: An application of the Attribute Hierarchy Method (Doctoral thesis). Available from ProQuest Dissertations and Theses database. (Publication No. 919011661)
Bardhoshi, G., & Erford, B. T. (2017). Processes and procedures for estimating score reliability and precision. Measurement and Evaluation in Counseling and Development, 50(4), 256–263.
Boora, S., Pasiphol, S., & Tangdhanakanond, K. (2015). Development of cognitive diagnostic testing on basic arithmetic operation. Procedia-Social and Behavioral Sciences, 191, 769–772.
Bradshaw, L. (2017). Diagnostic classification models. In A. A. Rupp & J. P. Leighton (Eds.), the handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 297–327). New York: Wiley Blackwell.
Brendefur, J. L., Johnson, E. S., Thiede, K. W., Strother, S., & Severson, H. H. (2018). Developing a multi-dimensional early elementary mathematics screener and diagnostic tool: The primary mathematics assessment. Early Childhood Education Journal, 46(2), 153–157.
Briggs, D. C., & Alonzo, A. (2012). The psychometric modeling of ordered multiple-choice item responses for diagnostic assessment with a learning progression. In A. C. Alonzo & A. W. Gotwals (Eds.), Learning progressions in science: Current challenges and future directions (pp. 293–316). Rotterdam: Sense Publishers.
Briggs, D. C., Alonzo, A., Schwab, C., & Wilson, M. (2006). Diagnostic assessment with ordered multiple-choice items. Educational Assessment, 11(1), 33–63.
Broaddus, A. E. (2011). An investigation into foundational concepts related to slope: An application of the Attribute Hierarchy Method (Doctoral thesis). Available from ProQuest Dissertations and Theses Global database. (Publication No. 3487353)
Chan, P. P., & Yan, K. Y. (2013). Alternative assessment: Developing e-portfolio for final year project. In S. K. S. Cheung, J. Fong, W. Fong, F. L. Wang, & L. F. Kwok (Eds.), International conference on hybrid learning and continuing education (pp. 90–101). Berlin: Springer.
Clements, D. H., & Sarama, J. (2004). Learning trajectories in mathematics education. Mathematical Thinking and Learning, 6(2), 81–89.
Cui, Y., & Leighton, J. P. (2009). The Hierarchy Consistency Index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46(4), 429–449.
De Champlain, A. F. (2010). A primer on classical test theory and item response theory for assessments in medical education. Medical Education, 44(1), 109–117.
Downing, S. M. (2004). Reliability: On the reproducibility of assessment data. Medical Education, 38(9), 1006–1012.
Earnest, D. (2015). When "half an hour" is not "thirty minutes": Elementary students solving elapsed time problem. In T. G. Bartell, K. N. Bieda, R. T. Putnam, K. Bradfield, & H. Dominguez (Eds.), Proceedings of the 37th annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (pp. 285–291). East Lansing, MI: Michigan State University.
Ebel, R. L., & Frisbie, D. A. (1991). Essentials of educational measurement (5th ed.). Englewood Cliffs: Prentice-Hall.
Fulmer, G. W. (2015). Validating proposed learning progressions on force and motion using the force concept inventory: Findings from Singapore secondary schools. International Journal of Science and Mathematics Education, 13(6), 1235–1254.
Gay, L. R., Mills, G. E., & Airasian, P. W. (2012). Educational research: Competencies for analysis and applications (10th ed.). Upper Saddle River, NJ: Merrill.
Gierl, M. J., Alves, C., & Taylor-Majeau, R. (2010). Using the Attribute Hierarchy Method to make diagnostic inferences about examinees’ knowledge and skills in mathematics: An operational implementation of cognitive diagnostic assessment. International Journal of Testing, 10(4), 318–341. https://​doi.​org/​10.​1080/​15305058.​2010.​509554.CrossRef
Gierl, M. J., Cui, Y., & Zhou, J. (2009). Reliability and attribute-based scoring in cognitive diagnostic assessment. Journal of Educational Measurement, 46(3), 293–313.
Hadenfeldt, J. C., Bernholt, S., Liu, X., Neumann, K., & Parchmann, I. (2013). Using ordered multiple-choice items to assess students’ understanding of the structure and composition of matter. Journal of Chemical Education, 90(12), 1602–1608.
Hadenfeldt, J. C., Neumann, K., Bernholt, S., Liu, X., & Parchmann, I. (2016). Students’ progression in understanding the matter concept. Journal of Research in Science Teaching, 53(5), 683–708.
Haladyna, T. M. (2004). Developing and validating multiple-choice test items (3rd ed.). New York: Routledge.
Herrera, S. G., Murry, K. G., & Cabral, R. M. (2012). Assessment accommodations for classroom teachers of culturally and linguistically diverse students. Boston, MA: Pearson Higher Education.
Hinton, P. R., McMurray, I., & Brownlow, C. (2014). SPSS explained. New York: Routledge.
Huff, K., Warner, Z., & Schweid, J. (2017). Large-scale standards-based assessments of educational achievement. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 399–476). New York: Wiley Blackwell.
Huntley, R., Welch, C. J. (1993, April). Numerical answer options: Logical or random order? Paper presented at the annual meeting of the American Educational Research Association, Atlanta, GA.
Kamii, C., & Russell, K. A. (2012). Elapsed time: Why is it so difficult to teach? Journal for Research in Mathematics Education, 43(3), 296–345.
Keehner, M., Gorin, J. S., Feng, G., & Katz, I. R. (2017). Developing and validating cognitive models in assessment. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 75–101). London: Wiley.
Lai, H., Gierl, M. J., & Babenko, O. (2015). Application of conditional means for diagnostic scoring. International Journal of Learning, Teaching and Educational Research, 12(3), 61–79.
Leighton, J. P., Cui, Y., & Cor, M. K. (2009). Testing expert-based and student-based cognitive models: An application of the Attribute Hierarchy Method and Hierarchy Consistency Index. Applied Measurement in Education, 22(3), 229–254.
Leighton, J. P., Gierl, M. J., & Hunka, S. M. (2004). The Attribute Hierarchy Method for cognitive assessment: A variation on Tatsuoka’s Rule-Space Approach. Journal of Educational Measurement, 41(3), 205–237.
Li, H., Song, N., Yang, T., & Xin, T. (2020). Development of assessment tool of learning progressions: Taking primary school students’ statistical thinking test for example. Journal of East China Normal University (Educational Sciences), 38(4), 72–82.
Nichols, P. D., Kobrin, J. L., Lai, E., & Koepfler, J. D. (2017). The role of theories of learning and cognition in assessment design and development. In A. A. Rupp & J. P. Leighton (Eds.), The handbook of cognition and assessment: Frameworks, methodologies, and applications (1st ed., pp. 41–74). New York: Wiley Blackwell.
Nortvedt, G. A., & Buchholtz, N. (2018). Assessment in mathematics education: Responding to issues regarding methodology, policy, and equity. ZDM Mathematics Education, 50(4), 555–570.
Pallant, J. (2016). SPSS survival manual: A step by step guide to data analysis using SPSS program (6th ed.). London: McGraw-Hill Education.
Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: The science and design of educational assessment. Washington, DC: National Academy Press.
Polit, D. F., & Beck, C. T. (2006). The content validity index: Are you sure you know what’s being reported? Critique and recommendations. Research in Nursing & Health, 29(5), 489–497.
Quaigrain, K., & Arhin, A. K. (2017). Using reliability and item analysis to evaluate a teacher-developed test in educational measurement and evaluation. Cogent Education, 4(1), 1–11.
Roberts, M. R., & Gierl, M. J. (2010). Developing score reports for cognitive diagnostic assessments. Educational Measurement: Issues and Practice, 29(3), 25–38.
Ross, A. (2017). Online assessments in math. In G. D. Allen & A. Ross (Eds.), Pedagogy and content in middle and high school mathematics (pp. 165–168). Rotterdam: Sense Publishers.
Shute, V. J., Leighton, J. P., Jang, E. E., & Chu, M. W. (2016). Advances in the science of assessment. Educational Assessment, 21(1), 34–59.
Sia, C. J. L., & Lim, C. S. (2018). Cognitive diagnostic assessment: An alternative mode of assessment for learning. In D. R. Thompson, M. Burton, A. Cusi, & D. Wright (Eds.), Classroom assessment in mathematics (pp. 123–137). Cham: Springer.
Sia, C. J. L., Lim, C. S., Chew, C. M., & Kor, L. K. (2019). Expert-based cognitive model and student-based cognitive model in the learning of “Time”: Match or mismatch? International Journal of Science and Mathematics Education, 17(6), 1–19.
Stacey, K., & Wiliam, D. (2012). Technology and assessment in mathematics. In M. Clements, A. Bishop, C. Keitel, J. Kilpatrick, & F. Leung (Eds.), Third international handbook of mathematics education (pp. 721–751). New York: Springer.
Szilágyi, J., Clements, D. H., & Sarama, J. (2013). Young children’s understandings of length measurement: Evaluating a learning trajectory. Journal for Research in Mathematics Education, 44(3), 581–620.
Tan, P. L., Lim, C. S., & Kor, L. K. (2017). Diagnosing primary pupils’ learning of the concept of" after" in the topic “Time” through knowledge states by using cognitive diagnostic assessment. Malaysian Journal of Learning and Instruction, 14(2), 145–175.
Tang, W. L., Tsai, J. T., & Huang, C. Y. (2020). Inheritance coding with Gagné-based learning hierarchy approach to developing mathematics skills assessment systems. Applied Sciences, 10(4), 1465–1483.
Tavakol, M., & Dennick, R. (2011). Post-examination analysis of objective tests. Medical Teacher, 33(6), 447–458.
Testa, I., Capasso, G., Colantonio, A., Galano, S., Marzoli, I., Scotti di Uccio, U., & Zappia, A. (2019). Development*** and validation of a university students’ progression in learning quantum mechanics through exploratory factor analysis and Rasch analysis. International Journal of Science Education, 41(3), 388–417.
Thompson, N. A. (2010). KR-20. In N. Salkind (Ed.), Encyclopedia of research design (pp. 667–668). Thousand Oaks: Sage.
Verschaffel, L., Van Dooren, W., & De Smedt, B. (2012). Mathematical learning. In N. M. Seel (Ed.), Encyclopedia of the sciences of learning (pp. 2107–2110). Boston, MA: Springer.
Wang, T. H. (2018). Developing a web-based assessment system for evaluating examinee’s understanding of the procedure of scientific experiments. EURASIA Journal of Mathematics, Science and Technology Education, 14(5), 1791–1801.
Wilson, M. (2005). Construct measures: An item response modelling approach. New Jersey, London: Lawrence Erlbaum Associates.
Wilson, M. (2009). Measuring progressions: Assessment structures underlying a learning progression. Journal of Research in Science Teaching, 46(6), 716–730.
Yusoff, M. S. B. (2019). ABC of content validation and content validity index calculation. Education in Medicine Journal, 11(2), 49–54.
Zhou, Z., Peverly, S. T., Boehm, A. E., & Chongde, L. (2000). American and Chinese children’s understanding of distance, time, and speed interrelations. Cognitive Development, 15(2), 215–240.
Metadaten
Titel
Development and validation of online cognitive diagnostic assessment with ordered multiple-choice items for ‘Multiplication of Time’
verfasst von
Huan Chin
Cheng Meng Chew
Hooi Lian Lim
Publikationsdatum
10.01.2021
Verlag
Springer Berlin Heidelberg
Erschienen in
Journal of Computers in Education / Ausgabe 2/2021
Print ISSN: 2197-9987
Elektronische ISSN: 2197-9995
DOI
https://doi.org/10.1007/s40692-020-00180-7

Weitere Artikel der Ausgabe 2/2021

Journal of Computers in Education 2/2021 Zur Ausgabe

OriginalPaper

Incorporating feedback in online cognitive diagnostic assessment for enhancing grade five students’ achievement in ‘time’

OriginalPaper

The effect of basic robotic coding in-service training on teachers’ acceptance of technology, self-development, and computational thinking skills in technology use

OriginalPaper

Flipping an EFL classroom with the LINE application: students’ performance and perceptions

OriginalPaper

Assessing computational thinking abilities among Singapore secondary students: a Rasch model measurement analysis

OriginalPaper

Effects of an interest-driven creation approach on students’ mobile learning performance and creativity in learning science in a science museum