Top

Journal of Science Education and Technology

Published in:

10-02-2021

Practices and Theories: How Can Machine Learning Assist in Innovative Assessment Practices in Science Education

Author: Xiaoming Zhai

Published in: Journal of Science Education and Technology | Issue 2/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

As cutting-edge technologies, such as machine learning (ML), are increasingly involved in science assessments, it is essential to conceptualize how assessment practices are innovated by technologies. To partially meet this need, this article focuses on ML-based science assessments and elaborates on how ML innovates assessment practices in science education. The article starts with an articulation of the “practice” nature of assessment both of learning and for learning, identifying four essential assessment practices: identifying learning goals, eliciting performance, interpreting observations, and decision-making and action-taking. I then extend a three-dimensional framework for innovative assessments, including construct, functionality, and automaticity, and based on which to conceptualize innovative assessments in three levels: substitute, transform, and redefine. Using the framework, I elaborate on how the 10 articles included in this special issue, Applying Machine Learning in Science Assessment: Opportunity and Challenge, advanced our knowledge of the innovations that ML brought to science assessment practices. I contend that the 10 articles exemplify a great deal of effort to transform the four components of assessment practices: ML allows assessments to target complex, diverse, and structural constructs, and thus better approaching the three-dimensional science learning goals of the Next Generation Science Standards (NGSS Lead States, 2013); ML extends the approaches used to eliciting performance and collecting evidence; ML provides a means to better interpreting observations and using evidence; ML supports immediate and complex decision-making and action-taking. I conclude this article by pushing the field to consider the underlying educational theories that are needed for innovative assessment practices and the necessities of establishing a “romance” between assessment practices and the relevant educational theories, which I contend are the prominent challenges to forward innovative and ML-based assessment practices in science education.

next article Comparison of Machine Learning Performance Using Analytic and Holistic Coding Approaches Across Constructed Response Assessments Aligned to a Science Learning Progression

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Author note. In the preparation of the manuscript From Substitution to Redefinition: A Framework of Machine Learning-based Science Assessment (hereby called ML Framework; Zhai et al., 2020a). I benefited from a long conversation with my co-author Mark Urban-Lurain, the co-founder of Automatic Analysis of Constructed Responses (AACR) group at Michigan State University, and Kevin Haudek, the current lead Principal Investigator of AACR. Urban-Lurain has dedicated more than 20 years in studying the use of technology in STEM education, particularly on constructed response assessments, and had valuable insights about multiple aspects of applying technology, such as machine learning (ML), in advancing STEM assessments. The conversation started from a comment Urban-Lurain made on how we should conceptualize “innovative assessments.” More specifically, he urged me to think why we could possibly use the word “redefinition” to portray “innovative assessment.” If assessment is still in forms of multiple-choice, constructed responses, etc., and teachers and students are involved in activities that they used to do, what do we mean by “innovative” to a degree that the assessment may be “redefined” by ML? This section provides information partially serving as a response to Urban-Lurain’ question.

Abd-El-Khalick, F., Boujaoude, S., Duschl, R., Lederman, N. G., Mamlok-Naaman, R., & Hofstein, A. (2004). Inquiry in science education: international perspectives. Science Education, 88(3), 397–419.CrossRef

Anderson, L. W., Krathwohl, D. R., & Bloom, B. S. (2001). A taxonomy for learning, teaching, and assessing: a revision of Bloom's taxonomy of educational objectives. Longman

Beatty, I. D., & Gerace, W. J. (2009). Technology-enhanced formative assessment: a research-based pedagogy for teaching science with classroom response technology. Journal of Science Education and Technology, 18(2), 146–162. https://doi.org/10.1007/s10956-008-9140-4.CrossRef

Bennett, R. E. (2018). Educational assessment: what to watch in a rapidly changing world. Educational Measurement: Issues and Practice, 37(4), 7–15CrossRef

Bennett, R. E., Deane, P., & van Rijn, W. P. (2016). From cognitive-domain theory to assessment practice. Educational Psychologist, 51(1), 82–107.CrossRef

Bertolini, R., Finch, S. J., & Nehm, R. H. (2021). Testing the impact of novel assessment sources and machine learning methods on predictive outcome modeling in undergraduate biology. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-020-09888-8.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74.

Chang, H.-Y., Quintana, C., & Krajcik, J. (2013). Using drawing technology to assess students’ visualizations of chemical reaction processes. Journal of Science Education and Technology, 23(3), 355–369. https://doi.org/10.1007/s10956-013-9468-2.CrossRef

Chi, M. T., Feltovich, P. J., & Glaser, R. (1981). Categorization and representation of physics problems by experts and novices. Cognitive Science, 5(2), 121–152CrossRef

Clark, R. E. (1983). Reconsidering research on learning from media. Review of Educational Research, 53(4), 445–459CrossRef

Clement, J. (2000). Model based learning as a key research area for science education. International Journal of Science Education, 22(9), 1041–1053CrossRef

Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52(4), 1–28CrossRef

Darling-Hammond, L. (2014). Next generation assessment: Moving beyond the bubble test to support 21st century learning. John Wiley & Sons

DeBoer, G. E., Quellmalz, E. S., Davenport, J. L., Timms, M. J., Herrmann-Abell, C. F., & Buckley, B. C. (2014). Comparing three online testing modalities: using static, active, and interactive online testing modalities to assess middle school students’ understanding of fundamental ideas and use of inquiry skills related to ecosystems. Journal of Research in Science Teaching, 51(4), 523–554.CrossRef

Duschl, R. (2008). Science education in three-part harmony: balancing conceptual, epistemic, and social learning goals. Review of Research in Education, 32(1), 268–291CrossRef

Ferrer-Torregrosa, J., Torralba, J., Jimenez, M. A., García, S., & Barcia, J. M. (2015). ARBOOK: development and assessment of a tool based on augmented reality for anatomy. Journal of Science Education and Technology, 24(1), 119–124. https://doi.org/10.1007/s10956-014-9526-4.CrossRef

Frezzo, D. C., Behrens, J. T., & Mislevy, R. J. (2010). Design patterns for learning and assessment: facilitating the introduction of a complex simulation-based learning environment into a community of instructors. Journal of Science Education and Technology, 19(2), 105–114. https://doi.org/10.1007/s10956-009-9192-0.CrossRef

Gale, J., Wind, S., Koval, J., Dagosta, J., Ryan, M., & Usselman, M. (2016). Simulation-based performance assessment: an innovative approach to exploring understanding of physical science concepts. International Journal of Science Education, 38(14), 2284–2302. https://doi.org/10.1080/09500693.2016.1236298.CrossRef

Gobert, J. D., & Pallant, A. (2004). Fostering students’ epistemologies of models via authentic model-based tasks. Journal of Science Education and Technology, 13(1), 7–22CrossRef

Harris, C. J., Krajcik, J. S., Pellegrino, J. W., & DeBarger, A. H. (2019). Designing knowledge-in-use assessments to promote deeper learning. Educational Measurement: Issues and Practice, 38(2), 53–67. https://doi.org/10.1111/emip.12253.CrossRef

Hickey, D. T., Taasoobshirazi, G., & Cross, D. (2012). Assessment as learning: enhancing discourse, understanding, and achievement in innovative science curricula. Journal of Research in Science Teaching, 49(10), 1240–1270.CrossRef

Jescovitch, L. N., Scott, E. E., Cerchiara, J. A., Merrill, J., Urban-Lurain, M., Doherty, J. H., & Haudek, K. C. (2020). Comparison of machine learning performance using analytic and holistic coding approaches across constructed response assessments aligned to a science learning progression. Journal of Science Education and Technology, 1–18

Kane, M. (2013). Validating the interpretations and uses of test scores. Journal of Educational Measurement, 50(1), 1–73CrossRef

Kelly, G. J., McDonald, S., & Wickman, P. O. (2012). Science learning and epistemology. In Second international handbook of science education (pp. 281–291). Springer

Kloser, M., Borko, H., Martinez, J. F., Stecher, B., & Luskin, R. (2017). Evidence of middle school science assessment practice from classroom-based portfolios. Science Education, 101(2), 209–231. https://doi.org/10.1002/sce.21256.CrossRef

Krajcik, J. S., & Mun, K. (2014). Promises and challenges of using learning technologies to promote student learning of science. Handbook of Research on Science Education, 2, 337–360

Lamb, R., Hand, B., & Kavner, A. (2020). Computational modeling of the effects of the science writing heuristic on student critical thinking in science using machine learning. Journal of Science Education and Technology, 1–15

Lee, H. S., Gweon, G. H., Lord, T., Paessel, N., Pallant, A., & Pryputniewicz, S. (2021). Machine learning-enabled automated feedback: supporting students’ revision of scientific arguments based on data drawn from simulation. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-020-09889-7.

Liaw, H., Yu, Y. R., Chou, C. C., & Chiu, M. H. (2020). Relationships between facial expressions, prior knowledge, and multiple representations: a case of conceptual change for kinematics instruction. Journal of Science Education and Technology, 1–12

Liu, O. L., Lee, H. S., & Linn, M. C. (2011). Measuring knowledge integration: validation of four-year assessments. Journal of Research in Science Teaching, 48(9), 1079–1107. https://doi.org/10.1002/tea.20441.CrossRef

Maestrales, S. Y., Zhai, X., Touitou, I., Schneider, B., & Krajcik, J. (2021). Using machine learning to evaluate multidimensional assessments of chemistry and physics. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-020-09895-9.

Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science teaching. In Examining pedagogical content knowledge (pp. 95–132). Springer

McMahon, D., Wright, R., Cihak, D. F., Moore, T. C., & Lamb, R. (2016). Podcasts on mobile devices as a read-aloud testing accommodation in middle school science assessment. Journal of Science Education and Technology, 25(2), 263–273. https://doi.org/10.1007/s10956-015-9591-3.CrossRef

Mercer-Mapstone, L., & Kuchel, L. (2015). Teaching scientists to communicate: evidence-based assessment for undergraduate science education. International Journal of Science Education, 37(10), 1613–1638. https://doi.org/10.1080/09500693.2015.1045959.CrossRef

Messick, S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Educational researcher, 23(2), 13–23CrossRef

Mislevy, R., & Haertel, G. (2006). Implications of evidence-centered design for educational testing. Educational measurement: issues and practice, 25(4), 6–20CrossRef

Mislevy, R. J. (2016). How developments in psychology and technology challenge validity argumentation. Journal of Educational Measurement, 53(3), 265–292CrossRef

National Research Council. (2012). A framework for K-12 science education: practices, crosscutting concepts, and core ideas. National Academies Press

Nehm, R. H., Ha, M., & Mayfield, E. (2012). Transforming biology assessment with machine learning: automated scoring of written evolutionary explanations. Journal of Science Education and Technology, 21(1), 183–196. https://link-springer-com.proxy1.cl.msu.edu/content/pdf/10.1007%2Fs10956-011-9300-9.pdf.

Neumann, K., & Waight, N. (2020). The digitalization of science education: Déjà vu all over again? Journal of Research in Science Teaching, 57(9), 1519–1528. https://doi.org/10.1002/tea.21668.CrossRef

NGSS Lead States. (2013). Next generation science standards: For states, by states. National Academies Press.

NGSS Lead States. (2018). Criteria for procuring and evaluating high-quality and aligned summative science assessments. https://www.nextgenscience.org/sites/default/files/Criteria03202018.pdf.

Nicolaidou, I., Kyza, E. A., Terzian, F., Hadjichambis, A., & Kafouris, D. (2011). A framework for scaffolding students’ assessment of the credibility of evidence. Journal of Research in Science Teaching, 48(7), 711–744CrossRef

Osborne, J. (2010). Arguing to learn in science: the role of collaborative, critical. Science, 1183944(463), 328

Osborne, J. F., Henderson, J. B., MacPherson, A., Szu, E., Wild, A., & Yao, S. Y. (2016). The development and validation of a learning progression for argumentation in science. Journal of Research in Science Teaching, 53(6), 821–846CrossRef

Pellegrino, J. W. (2013). Proficiency in science: assessment challenges and opportunities. Science, 340(6130), 320–323. https://science.sciencemag.org/content/340/6130/320.long.

Pellegrino, J. W. (2018). Sciences of learning and development: some thoughts from the learning sciences. Applied Developmental Science, 1–9

Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing what students know: the science and design of educational assessment. ERIC

Pellegrino, J. W., Wilson, M. R., Koenig, J. A., & Beatty, A. S. (2014). Developing assessments for the Next Generation Science Standards. ERIC

Penfield, R. D., & Lee, O. (2010). Test-based accountability: potential benefits and pitfalls of science assessment with student diversity. Journal of Research in Science Teaching, 47(1), 6–24CrossRef

Rosenberg, J. M., & Krist, C. (2020). Combining machine learning and qualitative methods to elaborate students’ ideas about the generality of their model-based explanations. Journal of Science Education and Technology, 1–13

Ruiz-Primo, M. A., & Furtak, E. M. (2007). Exploring teachers’ informal formative assessment practices and students’ understanding in the context of scientific inquiry. Journal of Research in Science Teaching, 44(1), 57–84CrossRef

Schwarz, C. V., Reiser, B. J., Davis, E. A., Kenyon, L., Achér, A., & Fortus, D. (2009). Developing a learning progression for scientific modeling: making scientific modeling accessible and meaningful for learners. Journal of Research in Science Teaching, 46(6), 632–654

Shavelson, R., Fu, A., Kurpius, A., & Wiley, E. (2013). Evidence- based practice in science education. In Encyclopedia of Science Education (pp. 1–4). https://doi.org/10.1007/978-94-007-6165-0_158-1.

Shavelson, R. J., Young, D. B., Ayala, C. C., Brandon, P. R., Furtak, E. M., Ruiz-Primo, M. A., & Yin, Y. (2008). On the impact of curriculum-embedded formative assessment on learning: a collaboration between curriculum and assessment developers. Applied measurement in education, 21(4), 295–314CrossRef

Shepard, L. A., Penuel, W. R., & Pellegrino, J. W. (2018). Using learning and motivation theories to coherently link formative assessment, grading practices, and large-scale assessment. Educational measurement: issues and practice, 37(1), 21–34CrossRef

Sung, S. H., Li, C., Chen, G., Huang, X., Xie, C., Massicotte, J., & Shen, J. (2020). How does augmented observation facilitate multimodal representational thinking? Applying deep learning to decode complex student construct. Journal of Science Education and Technology, 1–17

Wang, C., Liu, X., Wang, L., Sun, Y., & Zhang, H. (2020). Automated scoring of Chinese grades 7–9 students’ competence in interpreting and arguing from evidence. Journal of Science Education and Technology, 1–14

Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14CrossRef

Wilson, M. (2005). Constructing measures. An Item Response Modeling Approach. https://doi.org/10.4324/9781410611697.CrossRef

Yoo, J., & Kim, J. (2014). Can Online Discussion Participation Predict Group Project Performance? Investigating the Roles of Linguistic Features and Participation Patterns. International Journal of Artificial Intelligence in Education, 24(1), 8–32.CrossRef

Zhai, X. (2019). Applying machine learning in science assessment: Opportunity and challenges. A call for a special issue in Journal of Science Education and Technology. https://doi.org/10.13140/RG.2.2.10914.07365.

Zhai, X. (2021). Advancing automatic guidance in virtual science inquiry: from ease of use to personalization. Educational Technology Research and Development. https://doi.org/10.1007/s11423-020-09917-8.

Zhai, X., Haudek, K. C., Shi, L., Nehm, R., & Urban-Lurain, M. (2020a). From substitution to redefinition: a framework of machine learning-based science assessment. Journal of Research in Science Teaching, 57(9), 1430–1459. https://doi.org/10.1002/tea.21658.

Zhai, X., Haudek, K. C., Stuhlsatz, M. A., & Wilson, C. (2020b). Evaluation of construct-irrelevant variance yielded by machine and human scoring of a science teacher PCK constructed response assessment. Studies in Educational Evaluation, 67, 100916

Zhai, X., Krajcik, J., & Pellegrino, J. (2020c). On the validity of machine learning-based Next Generation Science Assessments: a validity inferential network. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-020-09879-9.

Zhai, X., Li, M., & Guo, Y. (2018). Teachers’ use of learning progression-based formative assessment to inform teachers’ instructional adjustment: a case study of two physics teachers’ instruction. International Journal of Science Education, 40(15), 1832–1856CrossRef

Zhai, X., Shi, L., & Nehm, R. (2020d). A meta-analysis of machine learning-based science assessments: factors impacting machine-human score agreements. Journal of Science Education and Technology. https://doi.org/10.1007/s10956-020-09875-z.

Zhai, X., Yin, Y., Pellegrino, J. W., Haudek, K. C., & Shi, L. (2020e). Applying machine learning in science assessment: a systematic review. Studies in Science Education, 56(1), 111–151

Zhang, M. (2015). Google photos Tags Two African-Americans As Gorillas Through Facial Recognition Software. Retrieved on January 3, 2021 from https://www.forbes.com/sites/mzhang/2015/07/01/google-photos-tags-two-african-americans-as-gorillas-through-facial-recognition-software/?sh=74cae6e1713d.

Title: Practices and Theories: How Can Machine Learning Assist in Innovative Assessment Practices in Science Education
Author: Xiaoming Zhai
Publication date: 10-02-2021
Publisher: Springer Netherlands
Published in: Journal of Science Education and Technology / Issue 2/2021
Print ISSN: 1059-0145
Electronic ISSN: 1573-1839
DOI: https://doi.org/10.1007/s10956-021-09901-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 2/2021

Machine Learning-Enabled Automated Feedback: Supporting Students’ Revision of Scientific Arguments Based on Data Drawn from Simulation

Using Machine Learning to Score Multi-Dimensional Assessments of Chemistry and Physics

Comparison of Machine Learning Performance Using Analytic and Holistic Coding Approaches Across Constructed Response Assessments Aligned to a Science Learning Progression

How Does Augmented Observation Facilitate Multimodal Representational Thinking? Applying Deep Learning to Decode Complex Student Construct

Commentary—Applying Machine Learning in Science Assessment: Opportunity and Challenges

Correction to: Combining Machine Learning and Qualitative Methods to Elaborate Students’ Ideas About the Generality of their Model-Based Explanations

Premium Partners