A framework to support educational decision making in mobile learning
Introduction
Learning Analytics is “the measurement, collection, analysis, and reporting of data about learners and their contexts, for the purposes of understanding and optimizing learning and the environments in which it occurs” (SoLAR, 2012). Learning Analytics developed originally in higher education and academic settings, as a shift from academic analytics and business intelligence towards learning processes.
Although Learning Analytics is considered to be a new research field – the definition reported above dates from 2012, and it is closely linked to a former definition by Siemens in 2010 (Siemens, 2010); the first international conference on Learning Analytics, LAK, was held in 2011 – Ferguson highlights how the origins of Learning Analytics date back to the twentieth century, and how this research area developed during the first decade of the new millennium (Ferguson, 2012). Important contributions to the definition of Learning Analytics have been provided by the field of Educational Data Mining (EDM), a data-driven approach to analyzing logs of student-computer interactions in order to support educators and learners (Zaïane, 2001). However, a determining and distinguishing factor in the construction of the definition of Learning Analytics has been the rise of learning-focused perspectives to Learning Analytics: in fact, over the last 10 years, socially and pedagogically driven approaches to analytics have led to educational applications which are strongly grounded in learning theories. The various research fields which have developed around the analysis of the data have therefore concentrated on different aspects: academic analytics has focused more on data analysis to improve administrative processes, while EDM has been more interested in the methodologies to use in analyzing the learning data, and the most recent Learning Analytics research has focused on the interpretation of the results aimed at guiding teachers in intervening to optimize learning processes.
From a pragmatic point of view, interest in Learning Analytics gathered pace with the rapid take-up of Virtual Learning Environments (VLEs) during the early 2000s. As a consequence of the diffusion of VLEs, especially in academic settings, a large amount of data concerning students’ learning patterns became available, initially attracting the attention of data mining specialists, and then the research interest of pedagogists, sociologists and educational technologists, thus fostering advancements in Learning Analytics mentioned above.
Central to the concept of Learning Analytics is therefore the possibility of logging and analyzing learner-produced data trails, as a collection of interactions between the learner and the learning context (Long & Siemens, 2011). Considering the development of Learning Analytics described above, it comes as no surprise to find that most applications of Learning Analytics have been characterized by well-structured and controlled learning contexts; these applications have focused on the exploitation of learner data stored in academic learning platforms, such as Learning Management Systems and Virtual Learning Environments used by students for fully distance or partially blended learning courses (Dyckhoff, Zielke, Bültmann, Chatti, & Schroeder, 2012), and on the different types of interactions occurring in these platforms (Johnson et al., 2008, Siemens, 2010, Richards, 2011, Agudo-Peregrina et al., 2014). Similarly, the learning resources with which the student interacted did not change during the learning experience (this only happened if planned from the beginning). On the one hand, this makes the management of data quite a straightforward process, but, on the other hand, it represents a formal learning situation which does not reflect the complexity of everyday learning.
Mobile and ubiquitous learning, MOOCs, social learning, Open Educational Resources (OER), Semantic Web and Linked Open Data (LOD) technologies are changing the way learning occurs: the number of learners in a MOOC as well as the technologies behind mobile and ubiquitous learning produce abundance of learner-produced data – also referred to as Big Data in the McKinsey Global Institute terminology (Manyika, 2011) – that generate datasets which are different both in size and type from the ones stored in VLEs (Merino et al., 2013, Ferguson, 2012); learners are increasingly engaged in informal learning settings, and social interactions are central to the participatory online culture which is reshaping the educational landscape (Ferguson and Buckingham Shum (2012)); OER and open data provide learners with dynamic sources of quality information on the Web, and Semantic Web technologies behind LOD support learners with a semantic layer in the interactions with educational context.
Accordingly, Learning Analytics approaches have to face these challenges in order to offer important insights to learners and teachers (Agudo-Peregrina et al., 2014, Ferguson, 2012). Important advances of Learning Analytics towards this objective can be found, among others, in the work of Ferguson and Buckingham Shum (2012), who propose the concept of Social Learning Analytics as a subset of Learning Analytics to specifically capture the social interactions underlying social learning processes; similarly, Aljohani and Davis (2012) point out the importance of a Learning Analytics model for mobile and ubiquitous learning environments. The potential cross-benefits of exploiting Linked Data as the data management layer for Learning Analytics are huge, even though the connections between the two research fields are not very well developed yet; thus the need emerges for a closer relationship between the fields of Learning Analytics and Linked Data (d’Aquin et al., 2013). In this perspective d’Aquin and Jay (2013) present an approach that exploits LOD in conjunction with the sequential pattern extraction method for the interpretation of insights resulting from the behavior of learners.
Drawing upon extant literature, this paper focuses on the challenge of using Learning Analytics techniques to support educational decision making in mobile learning settings. A task-interaction framework is presented with the aim of supporting teachers in assessing and evaluating learners during learning experiences based on mobile devices; it is rooted in the task model for mobile learners introduced by Taylor et al., 2006, Sharples et al., 2007, in the work on classification for mobile learning projects done by Frohberg, Göth, and Schwabe (2009) and in the classification of interactions for Learning Analytics proposed by Agudo-Peregrina et al. (2014). The framework considers three main steps, which are respectively related to: the pedagogical model underlying the mobile learning experience in a real learning context; the analysis of learner-produced data trails; the undertaking of specific actions to rearrange the learning activities according to evidence-based indicators (Fig. 1).
The paper has the following structure: Section 2 provides background about the impact of mobile devices on learning contexts with particular focus on the main factors and challenges related to the evaluation and assessment of mobile learning; furthermore, the potentials of using the Semantic Web to manage data in conjunction with Learning Analytics are introduced. In Section 3, the task-interaction framework is described. Then, a case study is presented in Section 4 in order to show an application of the framework to two learning scenarios. Considerations on the use of the framework are discussed in Section 5, and final remarks conclude the paper.
Section snippets
Background
Nowadays, sensors that are always on and mobile devices have been used more and more often to monitor activities in everyday life, collecting huge amounts of data about user behavior (Carmichael and Jordan, 2012, Duval, 2012). This trend also affects learning activities that take place anytime and anywhere. Thus, the challenges of evaluating Mobile Learning, a process which involves a number of independent variables that influence the learning process, are highlighted. Then, the state of the
The task-interaction framework
The framework to support educational decision-making in mobile learning introduced in this paper is based on the relationships between the different types of interactions occurring in a mobile learning activity and the tasks which are pedagogically relevant for the learning activity.
The framework has its roots in the task model for mobile learners introduced by Taylor et al., 2006, Sharples et al., 2007. Taylor and Sharples defined and developed their model to design and analyze a Mobile
Case study
A case study has been designed to demonstrate the application of the task-interaction framework to real learning scenarios based on the use of mobile devices.
Two mobile learning scenarios have been designed, in which mobile devices are used to support learning experiences on site. Even though the two scenarios both focus on learning activities carried out by high school students in a physical context, the two scenarios differ according to the curriculum adopted in the schools, the learning
Discussions
The case study has been introduced to demonstrate how the task-interaction framework can be used to support educational decision making in two mobile learning scenarios.
First of all, the learning experiences have been classified according to the six factors (context, tools, control, communication subject and objective) of the framework shown in Table 1.
Both scenarios, based on a learning experience that takes place on site, led us to consider the “physical context” as the most suitable value
Conclusions
Mobile Learning has reached a considerable level of maturity in recent years, and its role is widely acknowledged in school contexts, university, vocational training, formal and non-formal learning settings, and more generally as an opportunity for lifelong learning. Despite its maturity, evaluation of mobile learning remains an open research issue, especially as regards the activities that take place outside the classroom. In this context, Learning Analytics can provide answers, and offer the
References (46)
- et al.
Can we predict success from log data in VLEs? Classification of interactions for learning analytics and their relation with performance in VLE-supported F2F and online learning
Computers in Human Behavior
(2014) - et al.
Abductive science inquiry using mobile devices in the classroom
Computers & Education
(2013) - et al.
An empirical examination of factors contributing to the creation of successful e-learning environments
International Journal of Human-Computer Studies
(2008) - et al.
Mobile learning: Two case studies of supporting inquiry learning in informal and semiformal settings
Computers & Education
(2013) - et al.
Design patterns for monitoring and evaluating CSCL processes
Computers in Human Behavior
(2009) - et al.
Review of trends from mobile learning studies: A meta-analysis
Computers & Education
(2012) - Aljohani, N. R., & Davis, H. C. (2012). Learning analytics in mobile and ubiquitous learning environments. In...
- et al.
Enhancing social network analysis with a concept based text mining approach to discover key members on a virtual community of practice
Lecture Notes in Computer Science
(2010) - et al.
Meta-analyses from a collaborative project in mobile lifelong learning
British Educational Research Journal
(2013) - Bakharia, A., Dawson, S. (2011). SNAPP: A bird’s-eye view of temporal participant interaction. In Proceedings of the...
Linked data – The story so far. Special issue on linked data
International Journal on Semantic Web and Information Systems
Semantic web technologies for education – time for a ‘turn to practice’?
Technology, Pedagogy and Education
Design and implementation of a learning analytics toolkit for teachers
Educational Technology & Society
Learning by expanding: An activity-theoretical approach to developmental research
Learning analytics: drivers, developments and challenges
International Journal of Technology Enhanced Learning
Social learning analytics: five approaches
Mobile Learning projects—a critical analysis of the state of the art
Journal of Computer Assisted Learning
Cited by (79)
Implementing digital learning for health
2020, Digital Innovations in Healthcare Education and TrainingHands-on interaction in the augmented reality (AR) chemistry laboratories enhances the learning effects of low-achieving students: a pilot study
2024, Interactive Technology and Smart EducationMobile-assisted language learning and pronunciation instruction: A systematic literature review
2024, Education and Information TechnologiesMultimodal Teacher Dashboards: Challenges and Opportunities of Enhancing Teacher Insights Through a Case Study
2024, IEEE Transactions on Learning Technologies