Abstract
Peer Assessment is a powerful strategy to support educational activities and the consequent learners’ success. Learning performance of participating is often estimated in a peer assessment setting using Item Response Theory. In this paper, a feasibility of estimating individual performance is examined for a simulated data set representing a MOOC environment, where one thousand students are supposed to perform a Peer Assessment session, where each peer assesses three other peers’ work. For each student the modeling traits “ability”, “consistency”, and “strictness” are evaluated using Generalized Partial Credit Model, and the validity of such calculation is confirmed. While taking into consideration the limits of the synthetic sample production, this experiment provides an evidence of the possibility to predict learning performance in the large scale learning conditions of a MOOC.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Alcarria, R., Bordel, B., deAndrés, D.M., Robles, T.: Enhanced peer assessment in MOOC evaluation through assignment and review analysis. Int. J. Emerg. Technol. Learn. 13(1), 206–219 (2018)
Baker, F., Kim, S.H.: Item Response Theory: Parameter Estimation Techniques. Statistics, Textbooks and Monographs. Marcel Dekker, New York (2004)
Bloom, B.S.: Taxonomy of Educational Objectives. David McKay Company Inc., New York (1964)
De Marsico, M., Sciarrone, F., Sterbini, A., Temperini, M.: Supporting mediated peer-evaluation to grade answers to open-ended questions. EURASIA J. Math. Sci. Technol. Educ. 13(4), 1085–1106 (2017)
Fox, J.P.: Bayesian Item Response Modeling: Theory and Applications. Springer, Heidelberg (2010)
de Freitas, S.I., Morgan, J., Gibson, D.: Will MOOCs transform learning and teaching in higher education? engagement and course retention in online learning provision. Brit. J. Educ. Technol. 46(3), 455–471 (2015)
Gelman, A., Rubin, D.B.: Inference from iterative simulation using multiple sequences. Stat. Sci. 7(4), 457–472 (1992)
Hoffman, M.D., Gelman, A.: The No-U-Turn sampler: adaptively setting path lengths in Hamiltonian Monte Carlo. J. Mach. Learn. Res. 15, 1593–1623 (2014)
Jiang, Z., Carter, R.: Using Hamiltonian Monte Carlo to estimate the log-linear cognitive diagnosis model via Stan. Behav. Res. Methods 51(2), 651–662 (2019)
Lord, F.: Applications of Item Response Theory to Practical Testing Problems. Erlbaum Associates, Mahwah (1980)
Luo, Y., Jiao, H.: Using the stan program for bayesian item response theory. Educ. Psychol. Meas. 78(3), 384–408 (2018)
Mitchell, T.M.: Machine Learning, 1st edn. David McKay, New York (1997)
Muraki, E.: A generalized partial credit model. In: van der Linden, W.J., Hambleton, R.K. (eds.) Handbook of Modern Item Response Theory, pp. 153–164. Springer, Heidelberg (1997)
Patz, R.J., Junker, B.: Applications and extensions of MCMC in IRT: multiple item types, missing data, and rated responses. J. Educ. Behav. Stat. 24, 342–366 (1999)
Sciarrone, F., Temperini, M.: K-openanswer: a simulation environment to analyze the dynamics of massive open online courses in smart cities. Soft Computing (2020). In Press
Sciarrone, F., Temperini, M.: Simulating massive open on-line courses dynamics. In: Proceedings of iTHET 2019, Magdeburg, Germany, pp. 1–9 (2019)
Sun, D.L., Harris, N., Walther, G., Baiocchi, M.: Peer assessment enhances student learning: the results of a matched randmized crossover experiment in a college statistics class. PLoS ONE 10(12), 1–7 (2015)
Uto, M.: Rater-effect IRT model integrating supervised LDA for accurate measurement of essay writing ability. In: Proceedings of the International Conference on Artificial Intelligence in Education, pp. 494–506 (2019)
Uto, M., Ueno, M.: Item response theory for peer assessment. IEEE Trans. Learn. Technol. 9(2), 157–170 (2016)
Uto, M., Ueno, M.: Empirical comparison of item response theory models with rater’s parameters. Heliyon 4, 1–32 (2018)
Uto, M., Ueno, M.: Item response theory without restriction of equal interval scale for rater’s score. In: Proceedings of the International Conference on Artificial Intelligence in Education, pp. 363–368 (2018)
Acknowledgement
This research was partially supported by the Japan Society for the Promotion of Science (JSPS), KAKEN (17H00825).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nakayama, M., Sciarrone, F., Uto, M., Temperini, M. (2021). Estimating Student’s Performance Based on Item Response Theory in a MOOC Environment with Peer Assessment. In: Kubincová, Z., Lancia, L., Popescu, E., Nakayama, M., Scarano, V., Gil, A. (eds) Methodologies and Intelligent Systems for Technology Enhanced Learning, 10th International Conference. Workshops. MIS4TEL 2020. Advances in Intelligent Systems and Computing, vol 1236. Springer, Cham. https://doi.org/10.1007/978-3-030-52287-2_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-52287-2_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-52286-5
Online ISBN: 978-3-030-52287-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)