ABSTRACT
The effectiveness of using learning analytics for learning design primarily depends upon two concepts: grounding and alignment. This is the primary conjecture for the study described in this paper. In our design-based research study, we design, test, and evaluate teacher-facing learning analytics for an online inquiry science unit on global climate change. We design our learning analytics in accordance with a socioconstructivism-based pedagogical framework, called Knowledge Integration, and the principles of learning analytics Implementation Design. Our methodology for the design process draws upon the principle of the Orchestrating for Learning Analytics framework to engage stakeholders (i.e. teachers, researchers, and developers). The resulting learning analytics were aligned to unit activities that engaged students in key aspects of the knowledge integration process. They provided teachers with actionable insight into their students' understanding at critical junctures in the learning process. We demonstrate the efficacy of the learning analytics in supporting the optimization of the unit's learning design. We conclude by synthesizing the principles that guided our design process into a framework for developing and evaluating learning analytics for learning design.
- 2013. Next Generation Science Standards: For States, By States. National Academies Press, Washington, D.C. Google ScholarCross Ref
- Yoav Bergner, Geraldine Gray, and Charles Lang. 2018. What Does Methodology Mean for Learning Analytics? Journal of Learning Analytics 5, 2 (Aug. 2018), 1--8. Google ScholarCross Ref
- Ton de Jong. 2019. Moving towards engaged learning in STEM domains; there is no simple answer, but clearly a road ahead. Journal of Computer Assisted Learning 35, 2 (April 2019), 153--167. Google ScholarCross Ref
- Vanessa Echeverria, Roberto Martinez-Maldonado, Roger Granda, Katherine Chiluiza, Cristina Conati, and Simon Buckingham Shum. 2018. Driving data storytelling from learning design. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge - LAK '18. ACM Press, Sydney, New South Wales, Australia, 131--140. Google ScholarDigital Library
- Rebecca Ferguson. 2012. Learning analytics: drivers, developments and challenges. International Journal of Technology Enhanced Learning 4, 5/6 (2012), 304. Google ScholarDigital Library
- Dragan Gašević, Vitomir Kovanović, and Srećko Joksimović. 2017. Piecing the learning analytics puzzle: a consolidated model of a field of research and practice. Learning: Research and Practice 3, 1 (Jan. 2017), 63--78. Google ScholarCross Ref
- Pablo A. Haya, Oliver Daems, Nils Malzahn, Jorge Castellanos, and Heinz Ulrich Hoppe. 2015. Analysing content and patterns of interaction for improving the learning design of networked learning environments: Analysing content and patterns of interaction. British Journal of Educational Technology 46, 2 (March 2015), 300--316. Google ScholarCross Ref
- Davinia Hernández-Leo, Roberto Martinez-Maldonado, Abelardo Pardo, Juan A. Muñoz-Cristóbal, and María J. Rodríguez-Triana. 2019. Analytics for learning design: A layered framework and tools: Analytics layers for learning design. British Journal of Educational Technology 50, 1 (Jan. 2019), 139--152. Google ScholarCross Ref
- Claudia Leacock and Martin Chodorow. 2003. Crater: Automated Scoring of Short-Answer Questions. Language Resources and Evaluation - LRE 37 (2003), 389--405. Google ScholarCross Ref
- Hee-Sun Lee and Ou Lydia Liu. 2009. Assessing learning progression of energy concepts across middle school grades: The knowledge integration perspective. Science Education 94, 4 (Nov. 2009), 665--688. Google ScholarCross Ref
- Marcia C. Linn and Bat-Sheva Eylon. 2011. Science learning and instruction: taking advantage of technology to promote knowledge integration. Routledge, New York. OCLC: ocn176946367.Google Scholar
- Lori Lockyer, Elizabeth Heathcote, and Shane Dawson. 2013. Informing Pedagogical Action: Aligning Learning Analytics With Learning Design. American Behavioral Scientist 57, 10 (Oct. 2013), 1439--1459. Google ScholarCross Ref
- Katerina Mangaroska and Michail N. Giannakos. 2018. Learning analytics for learning design: A systematic literature review of analytics-driven design to enhance learning. IEEE Transactions on Learning Technologies (2018), 1--1. Google ScholarDigital Library
- Yishay Mor, Rebecca Ferguson, and Barbara Wasson. 2015. Editorial: Learning design, teacher inquiry into student learning and learning analytics: A call for action: Learning design, TISL and learning analytics. British Journal of Educational Technology 46, 2 (March 2015), 221--229. Google ScholarCross Ref
- Donatella Persico and Francesca Pozzi. 2015. Informing learning design with learning analytics to improve teacher inquiry: Informing LD with LA to improve teacher inquiry. British Journal of Educational Technology 46, 2 (March 2015), 230--248. Google ScholarCross Ref
- Luis P. Prieto, María Jesús Rodríguez-Triana, Roberto Martínez-Maldonado, Yannis Dimitriadis, and Dragan Gašević. 2018. Orchestrating learning analytics (OrLA): Supporting inter-stakeholder communication about adoption of learning analytics at the classroom level. Australasian Journal of Educational Technology (Nov. 2018). Google ScholarCross Ref
- Peter Reimann. 2016. Connecting learning analytics with learning research: the role of design-based research. Learning: Research and Practice 2, 2 (July 2016), 130--142. Google ScholarCross Ref
- María Jesús Rodríguez-Triana, Alejandra Martínez-Monés, Juan I. Asensio-Pérez, and Yannis Dimitriadis. 2015. Scripting and monitoring meet each other: Aligning learning analytics and learning design to support teachers in orchestrating CSCL situations: Scripting and monitoring meet each other. British Journal of Educational Technology 46, 2 (March 2015), 330--343. Google ScholarCross Ref
- María Jesüs Rodríguez-Triana, Luis P. Prieto, Alejandra Martínez-Monés, Juan I. Asensio-Pérez, and Yannis Dimitriadis. 2018. The teacher in the loop: customizing multimodal learning analytics for blended learning. In Proceedings of the 8th International Conference on Learning Analytics and Knowledge - LAK '18. ACM Press, Sydney, New South Wales, Australia, 417--426. Google ScholarDigital Library
- William A. Sandoval and Philip Bell. 2004. Design-Based Research Methods for Studying Learning in Context: Introduction. Educational Psychologist 39, 4 (Dec. 2004), 199--201. Google ScholarCross Ref
- Anouschka van Leeuwen. 2015. Learning analytics to support teachers during synchronous CSCL: Balancing between overview and overload. Journal of Learning Analytics 2, 2 (Dec. 2015), 138--162. Google ScholarCross Ref
- Keisha Varma, Freda Husic, and Marcia C. Linn. 2008. Targeted Support for Using Technology-Enhanced Science Inquiry Modules. Journal of Science Education and Technology 17, 4 (Aug. 2008), 341--356. Google ScholarCross Ref
- Tammie Visintainer and Marcia Linn. 2015. Sixth-Grade Students' Progress in Understanding the Mechanisms of Global Climate Change. Journal of Science Education and Technology 24, 2--3 (April 2015), 287--310. Google ScholarCross Ref
- Jonathan M. Vitale, Elizabeth McBride, and Marcia C. Linn. 2016. Distinguishing complex ideas about climate change: knowledge integration vs. specific guidance. International Journal of Science Education 38, 9 (June 2016), 1548--1569. Google ScholarCross Ref
- Korah J Wiley, Allison Bradford, and Marcia C. Linn. 2019. Supporting Collaborative Curriculum Customizations Using the Knowledge Integration Framework. In A Wide Lens: Combining Embodied, Enactive, Extended, and Embedded Learning in Collaborative Settings, 13th International Conference on Computer Supported Collaborative Learning, Kristine Lund, Gerald P. Niccolai, Elise Lavoue, Cindy Hmelo-Silver, Gahgene Gweon, and Michael Baker (Eds.), Vol. 1. International Society of the Learning Sciences (ISLS), Lyon, France, 480--487.Google Scholar
- Alyssa Wise, Yuting Zhao, and Simone Hausknecht. 2014. Learning Analytics for Online Discussions: Embedded and Extracted Approaches. Journal of Learning Analytics 1, 2 (2014), 48--71. Google ScholarCross Ref
- Alyssa Friend Wise and Jovita Vytasek. 2017. Learning Analytics Implementation Design. In Handbook of Learning Analytics (first ed.), Charles Lang, George Siemens, Alyssa Wise, and Dragan Gasevic (Eds.). Society for Learning Analytics Research (SoLAR), 151--160. Google ScholarCross Ref
Index Terms
- From theory to action: developing and evaluating learning analytics for learning design
Recommendations
A conceptual framework linking learning design with learning analytics
LAK '16: Proceedings of the Sixth International Conference on Learning Analytics & KnowledgeIn this paper we present a learning analytics conceptual framework that supports enquiry-based evaluation of learning designs. The dimensions of the proposed framework emerged from a review of existing analytics tools, the analysis of interviews with ...
Using learning analytics to explore help-seeking learner profiles in MOOCs
LAK '17: Proceedings of the Seventh International Learning Analytics & Knowledge ConferenceIn online learning environments, learners are often required to be more autonomous in their approach to learning. In scaled online learning environments, like Massive Open Online Courses (MOOCs), there are differences in the ability of learners to ...
Tracking transdisciplinary skills in the design of online courses: A design-based research study
AbstractThis design-based research (DBR) study was conducted to identify the type of learning activities and assessments that are responsible for fostering transdisciplinary skills (t-skills) in higher education online courses. T-skills are ...
Highlights- DBR identified the types of learning activities that foster transdisciplinary skills in higher education online courses.
Comments