Matching self-reports with electrodermal activity data: Investigating temporal changes in self-regulated learning

SRL emphasizes the pro-active role of individuals in completing academic tasks and focuses on the complex relationship between the cognitive, motivational, emotional, and behavioral regulatory components in the course of learning (Schunk and Zimmerman 2008). Decades of studies have reported that self-regulated learners apply effective strategies when setting learning goals, monitoring learning progress and adapting cognition, motivation, and behavior toward goal attainment, individually and with others (Schunk and Greene 2017). In collaborative learning contexts, research on SRL has been receiving more attention, such as research into the motivational and emotional aspects of collaboration (Isohätälä et al. 2017), but still mostly focused on the relationship between social interactions, cognitive processes, and their effects on knowledge construction and academic success (Hmelo-Silver and Barrows 2008). There is limited knowledge about the connections between the cognitive, behavioral, motivational, and emotional regulatory components of SRL and how they relate to learning outcomes in general (Efklides 2011), and collaborative learning in particular (Panadero and Järvelä 2015).

In educational contexts, behavioral regulation refers to sustaining attention, following instructions, and controlling actions to complete academic tasks (McClelland et al. 2007). Research has underlined the importance of behavioral regulation in adaptation to learning contexts and academic achievement (e.g. McClelland et al. 2006). On the other hand, lack of behavioral regulation as reflected in either activating or inhibiting a behavioral response was found to be detrimental to academic self-efficacy (Barkley 2004). Behavioral regulation also requires applying various cognitive skills (e.g., attention and working memory) to behavior (Sektnan et al. 2010). In this regard, a close relationship exists between behavioral and cognitive regulation.

Cognitive regulation involves processes such as activating prior knowledge, applying effective strategies to integrate new and existing knowledge, and evaluating understanding (Molenaar et al. 2011). Individuals obtain more domain knowledge if they regulate their cognition and engage in a greater number of cognitive activities during the learning activity (Pardo et al. 2017). In addition, social interaction is necessary for the activation of cognitive processes (Kreijns et al. 2003). Thus, collaborative learning environments have been regarded as appropriate grounds for developing cognitive structures and applying cognitive regulatory strategies through social interactions (e.g., asking questions, giving explanations, giving feedback, and argumentation) (Chi 2009).

It is well documented that focusing solely on cognitive processes is not sufficient for understanding individuals’ success or failure in naturalistic learning situations and that motivational regulation can substantially affect academic achievement (Schunk and Zimmerman 2008). Motivation refers to one’s willingness to engage with a task and persistence on activities towards task completion (Wolters 2003). Several studies have reported that motivational regulatory activities are related to enhanced cognitive and metacognitive strategy use and improved learning outcomes (e.g. Schwinger et al. 2009). Studies have further found that shared regulation of motivation among individuals in collaborative learning groups facilitated cognitive regulation processes in completing academic tasks (Järvelä et al. 2016a, b, c).

Even though the amount of research into various aspects of SRL has increased substantially in recent years, the role of emotional regulation in SRL and collaborative learning has remained an underexplored area of study (Järvenoja et al. 2013; Webster and Hadwin 2015). Emotions in the academic context can be called as intense reactions directed towards learning situation (Goetz et al. 2006). The limited research that exists has revealed that emotional regulation plays a critical role in coping with social challenges that arise during collaborative learning (Efklides 2011). It was assumed that emotional regulation during collaborative learning enhances interaction, trust, and engagement among learners and facilitates adaptive cognitive and behavioral strategies that might prompt SRL and enhance learning performance (Efklides and Volet 2005).

While the association between cognitive, motivational, and emotional regulatory processes in SRL is based on dynamic, complex, and reciprocal interactions (Pintrich 2004), current literature says little about how SRL processes develop together in collaborative learning and how the co-occurrence of such processes relates to learning outcomes. This is mostly due to methodological challenges in capturing SRL processes.

In recent decades, there has been a rapid evolution of measurement in the field of SRL (Panadero et al. 2016). This is mostly because conventional measurement tools, such as questionnaires, treated SRL constructs as relatively stable aptitudes (Winne and Perry 2000). However, the current conceptual and operational definitions of SRL characterizes it as an unfolding series of events that are affected by both individual and the contextual factors, such as nature of the task and the social environment (McCardle and Hadwin 2015).Therefore, self-reported questionnaires might not be able to capture the dynamicity of SRL components (Pintrich 2004).

Nonetheless, self-report data has a solid place in SRL research and offers several advantages (Perry and Winne 2006). It does not interfere with the learning process, is practical for large-scale data gathering, and the scoring of self-report data is straightforward and takes little time (Schellings and van Hout-Wolters 2011). In addition, understanding learners’ self-perceptions, self-evaluations, and reflections is critical for disclosing regulatory activities in SRL (Hadwin et al. 2011). Considering this, some scholars have asserted that the repeated utilization of single-item questionnaires can be useful in capturing on-task states and transitory learning processes (Ainley and Patrick 2006). Further, combining self-reports with process- oriented measures (e.g. physiological signals) might be a promising approach to match learners’ perceptions of SRL processes with their actual behaviors during learning (Azevedo et al. 2016).

Physiological data derived from the autonomic nervous system can provide objective information about real-time alterations in cognitive and affective states allowing research to go beyond what can be observed to reveal often-invisible cognitive and emotional reactions of the body and brain (Reimann et al. 2014). Heart rate variability (HRV) and electrodermal activity (EDA) are, for example, common measures of the autonomic nervous system that have been used to try to understand the cognitive and affective processes of individuals, such as cognitive load (Fairclough et al. 2005), emotional state (Calvo and D’Mello 2010), motivation and effort (Gendolla and Richter 2005), and attention (Ravaja 2004). In addition, there has been an interest in observing interpersonal autonomic physiology (i.e., physiological synchrony) between multiple individuals in active, social, and natural interaction situations in recent years (Palumbo et al. 2016).

Physiological synchrony (PS) is defined as “any interdependent or associated activity identified in the physiological processes of two or more individuals” (Palumbo et al. 2016, p. 2). Several indices have been developed to measure and describe PS. For example, Marci and Orr (2006) introduced a physiological concordance index derived from measures of EDA and found that PS was significantly higher in real therapy interactions than in hypothetical pairs and significantly correlated with self-reported empathy.

Only few studies have utilized PS measures in educational contexts. For example, Gillies et al. (2016) used wireless wristbands to measure EDA and analyzed the data in terms of PS between students during a science class. They concluded that the high-level common engagement during whole-class activities and student-centered learning during the collaborative group activities were reflected in the PS between the students. Ahonen et al. (2016) studied students executing a collaborative pair-programming task and found shared patterns in the students’ heart rate variability to be significantly associated with their self-reported workloads. Despite the significant amount of research on PS and physiological data, the research focusing on utilizing physiological data in educational contexts is still scarce. Still, studies have overall revealed that physiological data can be an indicative of behavioral, cognitive or affective processes during collaboration (Palumbo et al. 2016). The extensive literature on regulated learning also states that SRL is comprised of cognitive and affective processes. Considering this, it is worth investigating whether physiological data can inform about the dimensions of SRL as well. Nonetheless, physiological has been utilized to a very limited extend to investigate SRL (Azevedo et al. 2018).

A plethora of studies in the literature have investigated the interactions between behavioral, cognitive, motivational, and emotional processes of SRL and academic achievement separately (Mega et al. 2014). Until now, however, few empirical studies have tried to incorporate multiple processes into a single study and examine how those processes are interconnected with each other and with learning outcomes (e.g., Ben-Eliyahu and Linnenbrink-Garcia 2015; Mega et al. 2014). This may be due to possible challenges in applying conventional self-report measures over multiple episodes of learning. In the traditional sense, measuring different learning processes requires asking multiple questions for each process. Consequently, the length of any questionnaire increases as more processes are included in the study. Unfortunately, long questionnaires come with significant limitations in terms of measuring learning progress at multiple time points and at short intervals (Pintrich 2004). Considering such limitations, the current study employed single-item questionnaires to measure behavioral, cognitive, motivational, and emotional changes in collaborative learning repeatedly and unobtrusively.

The aim of this study is to examine the temporal changes of behavioral, cognitive, motivational, and emotional processes during collaborative learning and their relationship to PS and academic achievement. The research questions are as follows: 1) Are there any relationships between behavioral, cognitive, motivational, and emotional regulatory processes and academic achievement?; 2) Are there any relationships between the PS of students and their self-reports about behavioral, cognitive, motivational, and emotional change during learning sessions?; 3) Is there any relationship between the PS of students and their academic success?

Participants in the study were 31 (23 males, 8 females) Finnish high school students, whose ages ranged between 15 and 16 years. The students enrolled in an Advanced Physics course consisting of 15 lessons. The students were divided into 10 heterogeneous groups based on their previous grades in order to avoid clustering of high and low achieveing students in the same groups. The groups comprised three (9 groups) to four members (1 group), and the students collaborated in the same groups in each lesson. Due to limitations in resources, only the 12 students in randomly chosen four groups given Empatica 4.0 (Empatica Inc., Boston, MA) wristbands for EDA measurement. To make the situation as similar as possible for all of the students, other six groups in the classroom (19 students) were asked to wear standard Polar Active activity monitors during the lessons. The data in the current study was collected with the authorization of the ethics committee at the university of the first author. Participation to the study was voluntary and no incentive was offered for participation.

Each of the physics lessons involved collaborative learning guided by an EdX online learning environment (https://​www.​edx.​org/​). The online environment further served as a data collection tool and asked students to evaluate their behavior, cognition, motivation, and emotion individually at the beginning and the end of the learning sessions.

Each learning session in the current study was organized as follows: 1) Students entered the classroom and sat at the same table as their group members. All students were given tablet computers and were asked to access the EdX learning environment with their tablet. 2) Teacher explained the theoretical background of the topic. 3) Students were presented with a collaborative learning task. Collaborative learning tasks consisted of conducting hands-on experiments (e.g. Finding out the circumstances convex mirror makes a virtual picture by using a specific simulation in the EdX environment), or paper-and-pencil problems (e.g. Designing an experimental setting on which one could possibly measure the speed of light. Drawing a picture about the setting with argumentation). 4) Following the task presentation, all students were asked to answer a pre-test questionnaire individually. 5) Students worked as groups to complete the collaborative learning task. 6) After completing the task, all students were asked to answer a post-test questionnaire individually. 7) Students left the classroom.

In the data analysis, first, self-reported data and temporal changes in behavior, cognition, motivation, and emotion were investigated and then examined to compare if and how the changes correlated with learning outcomes. The correlational analysis about the relationship between SRL changes and group exam scores was left out due to low amount of variation in group exam scores. Second, the PS of the dyads in the collaborating groups was determined by calculating a single session index (SSI) of physiological concordance for each session. Third, the correlation between PS, learning outcomes, and the temporal dimension changes in behavior, cognition, motivation, and emotion were investigated. Finally, the connection was investigated between PS and self-reported changes in behavior, cognition, motivation, and emotion.

A Pearson’s correlation was calculated to examine how overall behavioral, cognitive, motivational, and emotional changes were related to each other and the learning outcomes. The results are displayed in Table 4.

The results show that there was a small-to-moderate correlation between overall motivational change and end-of-term scores. By contrast, behavioral, cognitive and emotional regulation were not related with any of the learning outcomes. The results in Table 4 also revealed that cognitive change was correlated only with behavioral change and that emotional change was only correlated with motivational change. Motivational change was, however, correlated with behavioral change. The magnitude of the significant correlations can be considered as small to medium. The inclusion or exclusion of the outlier case in motivational change did not affect the significance of the findings.

To investigate the relationship between PS and changes in SRL processes, first, the average of each dyad’s self-reported change for every SRL dimension across the sessions was calculated. Second, academic achievement scores of the dyads were defined by calculating the average of the scores for the individual exam and end-of-term scores. Third, Spearman’s rho correlation formula with SPSS21 software was applied to answer our second research question. The results showed that the only significant relationship was between PS and cognitive change (r = .642) in terms of SRL dimensions. No significant relationship was observed between PS and any of the academic achievement scores.

To investigate the interplay of PS and self-reported data further, for each SRL process, independent samples t-tests were conducted to see whether there was a difference between the PS scores of dyads who had concordance in their self-reported SRL processes and those who did not have concordance. A significant difference was observed at behavioral (t_(1,32) = 2.044; p = .049; η_p² = .115), cognitive (t_(1,32) = 2.137; p = .040; η_p² = .125), and motivational dimensions (t_(1,32) = 3.866; p = .001; η_p² = .318). No difference was observed for the emotional dimension (t _{(1, 32)} = −.174; p > .05; η_p² = .001). Findings indicated that PS was significantly higher when there was concordance between the changes in dyads’ self-reported behavioral, cognitive, and motivational processes (see Fig. 1).

Based on the self-report data here, students’ behavioral change was positively correlated with their cognitive and motivational changes. In the context of SRL, the term behavioral processes refers to executive control in maintaining attention to a task and inhibiting unnecessary actions that might interfere with the completion of the task (McClelland et al. 2007) requiring conscious control of thoughts and actions (Happaney et al. 2004). Cognitive processes can be considered an important companion of behavioral processes. The moderate correlation between behavioral and cognitive changes in the current study supports this assumption. Behavioral processes further incorporate time and effort management and the seeking of help in collaborative learning environments (Pintrich 2004). In this regard, behavioral processes involve elements of persistence and determination. Supporting such claims, the correlation between behavioral and motivational changes in the current study indicated that behavioral management of collaborative learning processes are related to motivational change. By contrast, no significant correlation was observed between behavioral change and emotional change in the current study, nor was any found in previous studies. Although there was no direct relationship found between behavioral and emotional change, it is possible that their relationship may be mediated by motivational or cognitive processes. Therefore, rejection of any relationship between behavioral and emotional change seems to be premature. Unfortunately, the association between behavioral and emotional regulation has been so far neglected in SRL research. Further studies are necessary to develop a better understanding of the relationship between behavioral and emotional processes.

Previous studies have shown that emotions are associated with cognitive and motivational processes in SRL (e.g. Efklides 2011; Mega et al. 2014). Some scholars have asserted that positive emotions lead to effective use of cognitive and motivational strategies when completing academic tasks (Efklides and Volet 2005; Järvenoja et al. 2013). The findings here partially support this claims. In the sample, motivational change was significantly correlated with emotional change, whereas cognitive change was not. It should be also noted that there was no significant correlation between cognitive and motivational change. These findings do not support the general understanding in the SRL field that emotional and motivational regulatory processes enhance cognitive regulatory activities (Järvelä et al. 2016a, b, c; Malmberg et al. 2015; Schunk and Zimmerman 2008).

Behavioral, cognitive and emotional change did not correlate with any of the academic achievement scores. By contrast, motivational change correlated with end-of-term scores. With regard to behavioral change, some scholars have reported a significant relationship between behavioral regulation and academic success (McClelland et al. 2007) whereas others could not find any direct relationship between behavioral regulation and academic success (Ben-Eliyahu and Linnenbrink-Garcia 2015). Our findings here are in line with those of the latter group of scholars. However, the attributes of the research design here had specific differences from those of other studies. Ben-Eliyahu and Linnenbrink-Garcia (2015) measured behavioral regulation as a trait without focusing on SRL in collaborative learning contexts, and Janssen et al. (2012) coded student activities during collaborative work. Considering that all the learning activities took part in a collaborative format, in the current study we asked students to report their expectations and evaluations about how the group would and did perform during each collaborative learning session. The findings here showed that students’ evaluations of group-level behavioral changes did not affect their individual academic achievement.

With regard to cognitive change, several studies have reported a positive relationship between cognitive regulation and academic success (Chi 2009). However, other studies found no direct relationship between cognitive regulation and achievement (Ben-Eliyahu and Linnenbrink-Garcia 2015; Janssen et al. 2012) which was also the case here. Ben-Eliyahu and Linnenbrink-Garcia (2015) also found that cognitive regulation may predict academic achievement through mediation of learning strategies and engagement. As in Ben-Eliyahu and Linnenbrink-Garcia (2015), it is possible that cognitive processes interacted with academic achievement scores through other variables, such as learning strategies and engagement, in the current study.

Past studies have found that positive emotions enhanced cognitive and motivational regulatory processes and eventually led to academic success (Ahmed et al. 2013; Goetz et al. 2006). Similarly, motivational regulation was found to be influential on activation of cognitive and metacognitive strategies and successful task completion (Malmberg et al. 2015; Schwinger et al. 2009). Partly supporting the previous studies, the findings of the current study showed motivational changes to be significantly related to learning outcomes. However no direct relationship was found between emotional regulation and academic success.

There was a significantly positive relationship between cognitive change and PS. However, the correlations between PS and other changes (i.e., behavior, motivation, and emotion) were not significant. According to the literature, EDA can be an indicator of motivational, emotional, or cognitive arousal (Palumbo et al. 2016). Moreover, some scholars have reported that PS was not dependent on behavioral regulation (Henning et al. 2001). In terms of behavioral and cognitive change, our findings support previous studies. The lack of connection between PS and emotional change might be explained by the fact that self-report of emotional processes in this study focused more on valence dimension of emotion instead of arousal.

The findings further revealed that PS values were higher in collaborating pairs for the learning sessions in which the self-reported data showed concordance among the pairs in terms of behavioral, cognitive, and motivational changes. Specifically, if the perceived behavioral, cognitive, or motivational changes of the pairs were in the same direction (either increased or decreased together), the PS between the students was higher. These findings indicate that physiological signals can serve as a possible triangulation tool for SRL and collaborative learning research.

Finally, no relationship was observed between PS and academic achievement. The findings do not corroborate several previous studies that found a relationship between PS and performance (Elkins et al. 2009; Walker et al. 2012). On the other hand, in some studies, the relationship between performance and PS was not significant when task conditions were controlled (Montague et al. 2014). Therefore, one can assume that the relationship between PS and academic achievement may be context or task dependent.