Elsevier

Computers & Education

Volume 87, September 2015, Pages 204-217
Computers & Education

Learning at distance: Effects of interaction traces on academic achievement

https://doi.org/10.1016/j.compedu.2015.07.002Get rights and content

Highlights

  • We examined the relationship between interaction types and learning out-come.

  • The findings show significant positive effect of student–system interactions.

  • Student-content interactions were negatively associated with the learning outcome.

  • Educational level and course context are important for interaction types supported.

Abstract

Contemporary literature on online and distance education almost unequivocally argues for the importance of interactions in online learning settings. Nevertheless, the relationship between different types of interactions and learning outcomes is rather complex. Analyzing 204 offerings of 29 courses, over the period of six years, this study aimed at expanding the current understanding of the nature of this relationship. Specifically, with the use of trace data about interactions and utilizing the multilevel linear mixed modeling techniques, the study examined whether frequency and duration of student–student, student–instructor, student–system, and student–content interactions had an effect of learning outcomes, measured as final course grades. The findings show that the time spent on student–system interactions had a consistent and positive effect on the learning outcome, while the quantity of student–content interactions was negatively associated with the final course grades. The study also showed the importance of the educational level and the context of individual courses for the interaction types supported. Our findings further confirmed the potential of the use of trace data and learning analytics for studying learning and teaching in online settings. However, further research should account for various qualitative aspects of the interactions used while learning, different pedagogical/media features, as well as for the course design and delivery conditions in order to better explain the association between interaction types and the learning achievement. Finally, the results might imply the need for the development of the institutional and program-level strategies for learning and teaching that would promote effective pedagogical approaches to designing and guiding interactions in online and distance learning settings.

Introduction

With the development of technology, distance and online education provides a wide spectrum of interactive learning opportunities (Bernard et al., 2009, Bouhnik and Marcus, 2006, Donnelly, 2010, Woo and Reeves, 2007). Over the past few decades, interaction – as a main component of distance and online learning – has been studied by various researchers (e.g., Anderson, 2003, Arbaugh and Benbunan-Fich, 2007, Bernard et al., 2004, Wagner, 1994), commonly using Moore's (1989) framework of interactions (e.g., Agudo-Peregrina et al., 2014, Anderson, 2003, Kanuka, 2011). According to those considerations, learning occurs when a student interacts with other students or with an environment regardless of a subject domain, instructional design or the technology used in the learning process (Tirri & Kuusisto, 2013). Many researchers consider interaction as the most important component of any learning environment (Woo & Reeves, 2007), and thus, importance of interactions in both traditional (e.g., Tirri and Kuusisto, 2013, Mehan, 1998, Johnson, 1981, Yee, 1971) and distance and online educational settings (e.g., Anderson, 2003, Bernard et al., 2009, Hirumi, 2002, Lou et al., 2006, Moore, 1989, Muirhead and Juwah, 2005, Wagner, 1994, Woo and Reeves, 2007) have been studied for a long period of time.

Despite a prevalent understanding of the importance of interaction in online education, research literature does not recognize a unique definition of interaction. Interaction is rather defined from various perspectives, within different contexts, based on the participants involved and the level of their engagement (Bernard et al., 2009, Woo and Reeves, 2007). Wagner (1994) looks at interaction from the functional perspective, as an emerging process that involves communication in various forms. Moreover, she argues that each interaction contains at least two complementary, interrelated, events that occur between two objects. Wagner (1994) also notes that the goal of interaction is to change a student's educational behavior and to bring the student closer to the learning goal. On the other hand, Yacci (2000) defines interactivity as a loop of mutually coherent messages, that should complete the cycle (from and to the student) in order for interaction to occur. The final interaction outcome is either learning of some content or affective benefits. Yacci (2000) further argues for existence of the student-centered perspective to interactivity, which means that students will not confirm the existence of interaction unless they obtain some feedback. Yacci (2000) suggests a communication theory as a valid framework for analyzing online interactions, which includes a wide variety of variables (e.g., the count and length of messages, the type of information and the amount of time spent between two messages) that should be considered when analyzing online interaction. Further, building on the previous definitions of Yacci (2000) and Wagner, 1994, Muirhead and Juwah, 2005 developed a similar understanding of interaction. According to their definition, interaction represents an event (i.e., communication in any possible form) that occurs between two or more subjects (participants or objects). It might occur synchronously or asynchronously utilizing technology and providing response or feedback as an outcome. Muirhead and Juwah (2005) also recognize the need to differentiate interactions depending on the context in which they occur (e.g., proactive inquiry, reactive inquiry, proactive elaboration).

The majority of the studies that analyzed interactions in online and distance education relied on a perceived measures of interaction (Bernard et al., 2009, Borokhovski et al., 2012). While being useful, those measures are not always suitable, especially given the survey fatigue that is well documented in the literature (Ben-Nun, 2008) and the availability of massive amount of trace data logged by various educational platforms (Phillips, Maor, Preston, & Cumming-Potvin, 2012). Therefore, this paper offers insights into how the methods of learning analytics (Siemens & Gašević, 2012) can be used to study effects of interaction on learning in distance and online education. Specifically, the study reported in this paper aims at investigating (i) the extent to which the trace data can be used to measure the interaction types as theorized in contemporary research in distance and online education, (ii) the effects of these measures of the interaction types on learning success; and (iii) whether the effects of interactions types differ across different courses while students are progressing toward their academic degrees (from foundational to core and elective disciplinary courses).

Section snippets

Interactions in distance education

The conceptual framework developed by Moore (1989) identifies three types of interactions: i) student–content, ii) student–instructor, and iii) student–student. The student–content interaction type represents the essence of education (Moore, 1989) and identifies the relation that occurs between a student and the content that describes the subject of studying. More recently, researchers and course designers, following the social constructivistic principles, suggest that content is distributed

Method

In this section, we describe the data collection process and measures used in the study. Moreover, we explained the procedure followed to conduct the study and the analysis method performed on the collected data.

Results

The final model included course group, course within a course group, count of student–content interactions, time spent on student–system interactions, as well as the interaction effect between course group and time on student–teacher interactions, and count of student–student interactions (Table 4). A comparison of AICc values for the null (AICc = 1224.8) and the chosen model (AICc = 1159.7) provided a strong evidence favoring the model that included fixed and random effects (Burnham &

Interpretation of the results with respect to the research questions

The results of our study further contribute to the understanding of importance of the interactions in an online and distance educational settings (e.g., Bernard et al., 2009, Lou et al., 2006, Muirhead and Juwah, 2005, Woo and Reeves, 2007). Moreover, we also revealed that educational level and context of a particular course have a significant impact on interaction types supported, and therefore on their importance for the student achievement.

Among analyzed interaction types, time spent on

References (90)

  • G. Lust et al.

    Content management systems: enriched learning opportunities for all?

    Computers in Human Behavior

    (2012)
  • L.P. Macfadyen et al.

    Mining LMS data to develop an “early warning system” for educators: a proof of concept

    Computers & Education

    (2010)
  • J. Ma et al.

    Examining the necessary condition for engagement in an online learning environment based on learning analytics approach: the role of the instructor

    The Internet and Higher Education

    (2015)
  • L.V. Morris et al.

    Tracking student behavior, persistence, and achievement in online courses

    The Internet and Higher Education

    (2005)
  • C. Ramos et al.

    “Hits” (not “Discussion Posts”) predict student success in online courses: a double cross-validation study

    Computers & Education

    (2008)
  • C. Romero et al.

    Data mining in course management systems: moodle case study and tutorial

    Computers & Education

    (2008)
  • B. Rubin et al.

    The effect of learning management systems on student and faculty outcomes

    The Internet and Higher Education

    (2010)
  • R.F. Schmid et al.

    The effects of technology use in postsecondary education: a meta-analysis of classroom applications

    Computers & Education

    (2014)
  • S. Schrire

    Knowledge building in asynchronous discussion groups: going beyond quantitative analysis

    Computers & Education

    (2006)
  • Y. Woo et al.

    Meaningful interaction in web-based learning: a social constructivist interpretation

    The Internet and Higher Education

    (2007)
  • T. Anderson

    Getting the mix right again: an updated and theoretical rationale for interaction

    The International Review of Research in Open and Distance Learning

    (2003)
  • Y.M.R. Archila

    Interaction in a blended environment for English language learning

    GIST Education and Learning Research Journal

    (2014)
  • P. Ben-Nun

    Respondent fatigue

    (2008)
  • R.M. Bernard et al.

    A meta-analysis of three types of interaction treatments in distance education

    Review of Educational Research

    (2009)
  • R.M. Bernard et al.

    How does distance education compare with classroom instruction? a meta-analysis of the empirical literature

    Review of Educational Research

    (2004)
  • P. Blikstein

    Using learning analytics to assess students' behavior in open-ended programming tasks

  • E. Borokhovski et al.

    Are contextual and designed student–student interaction treatments equally effective in distance education?

    Distance Education

    (2012)
  • D. Bouhnik et al.

    Interaction in distance-learning courses

    Journal of the American Society for Information Science and Technology

    (2006)
  • A. Bryman

    Social research methods

    (2012)
  • K.P. Burnham et al.

    Model selection and multimodel inference: A practical information-theoretic approach

    (2002)
  • C. Carroll et al.

    UK health-care professionals' experience of on-line learning techniques: a systematic review of qualitative data

    Journal of Continuing Education in the Health Professions

    (2009)
  • A. Clardy

    Distant, on-line education: Effects, principles and practices

    (2009)
  • R. Clark

    Media will never influence learning

    Educational Technology Research and Development

    (1994)
  • R.E. Clark

    Confounding in educational computing research

    Journal of Educational Computing Research

    (1985)
  • R.E. Clark

    Evaluating distance education: strategies and cautions

    Quarterly Review of Distance Education

    (2000)
  • A. Darabi et al.

    Effectiveness of online discussion strategies: a meta-analysis

    American Journal of Distance Education

    (2013)
  • M. De Laat

    Networked learning

    (2006)
  • J. Eckles et al.

    A social network analysis of student retention using archival data

    Social Psychology of Education

    (2012)
  • L. Fang et al.

    Analyzing binomial data in a split-plot design: classical approach or modern techniques?

    Communications in Statistics: Simulation and Computation

    (2012)
  • A.P. Field et al.

    How to design and report experiments

    (2003)
  • N. Friesen et al.

    Modes of interaction

  • D.R. Garrison et al.

    Facilitating cognitive presence in online learning: interaction is not enough

    American Journal of Distance Education

    (2005)
  • D. Gašević et al.

    “Choose your classmates, your GPA is at stake!”: the association of cross-class social ties and academic performance

    American Behavioral Scientist

    (2013)
  • C.N. Gunawardena et al.

    Analysis of a global online debate and the development of an interaction analysis model for examining social construction of knowledge in computer conferencing

    Journal of Educational Computing Research

    (1997)
  • T. Hastie et al.

    The elements of statistical learning

    (2009)
  • Cited by (0)

    View full text