1 Introduction
In recent decades, we have witnessed the emergence of technologies that have turned into tangible cultural artifacts that mediate several aspects of our daily activities. Today, information and communication technologies such as computers and mobile technology are—in addition to being integrated into our leisure activities—tools that support productivity and efficiency and that in a fundamental way change how we acquire and produce knowledge (Jewitt 2008). In fact, knowledge acquisition takes place today through constant access to a global digital knowledge bank and people contribute to the production of knowledge more than ever before in human history.
This great revolutionary change is, to paraphrase Kress (2003), of semiotic kind. Today, knowledge communication, acquisition, and construction is done with support of multimodal affordances of new technologies (Conole and Dyke 2004). That is to say, knowledge is now more frequently represented and communicated in other modalities, such as audio, video, image, or a combination of these, than in the historically dominant text mode (Jewitt 2008).
Anzeige
Consequently, information and communication technology increasing integration in our lives and its profound effect on the communication landscape emphasizes the growing importance of digital literacy (Chan et al. 2017; Jewitt 2008; Kress and van Leeuwen 2001) and multimodal learning in both formal and informal settings (Selander 2016; Selander and Kress 2010; Jewitt 2008; Kress 2003). Kress (2003), for example, argues that new media practices brought about by new technologies produce a profound shift in education in a social as well as a pedagogical sense, resulting in the expansion of the traditional literacy concept to encompass communications involving new semiotic modes. With the support of Jewitt and Kress (2003), Leander and Lewis (2008) also argue that information and communication technologies (ICT) impose demands on learners to create meaning laterally across modes.
The emergence and integration of ICT and accompanying multimodal learning has also had an impact on higher education. As Sankey et al. (2010) point out, we have seen an “increasing use of multimedia in higher education teaching that has provided many opportunities to present multiple representations of content (text, video, audio, images) to cater more effectively to the different learning styles and modal preferences of an increasingly diverse student body” (p. 853).
In tune with contemporary neuroscience research which has revealed that multimodal learning is beneficial (Fadel and Lemke 2012), several studies have shown that students are more comfortable and perform better in multimodal learning environments that cater for their predominant learning style (Paxton et al. 2017; Kharb et al. 2013; Cronin 2009). From a review of research of multimodal learning in higher educational contexts, the following three strands of research have been distinguished.
-
First, a strand of research has investigated multimodal teaching and instruction that puts a particular emphasis on students consuming teaching materials that are based on other modes of representation or communication than the traditional text mode. In general, this entails students being consumers of teacher made material within course work, which can be exemplified by Roberts (2017) who focused on the use of images in teaching and learning activities, Walker et al. (2011) who investigated the use of video and audio recordings to enhance student learning in an introductory biology class, or Costley et al. (2016) who looked at students’ engagement when learning trough video in online environments.
-
Second, a strand of research has investigated the students’ production of learning material through the use of single modes of representation or communication. Examples of this research can be found in Liu (2016) who investigated how video blogs can support learning, or Lee et al. (2008) who cast light on student generated audio podcasts for collaborative learning purposes.
-
Third, a strand of research has investigated the design of multimodal learning environments, mostly in the frame of learning management systems. This can be exemplified by Song et al. (2016) or Moreno and Mayer’s (2007) literature review, which examines experimental studies on multimodal learning environments and which derives design principles for such environments.
While these research efforts have contributed to our understanding of multimodal learning in higher education, several interesting and important aspects have been overlooked. For instance, these research strands all focus on multimodal practices that are orchestrated by a teacher and that mainly take place on campus or in a formal digital learning environment. Thus, none of these research efforts have shed light on the university students’ unforced enactment of multimodal learning practices during self-studies. This gap of research was also noted by Lea and Jones (2011) who called for detailed examinations of how students in the digital age, and outside teacher orchestrated classroom environments, engage in learning from a multimodal perspective.
Anzeige
While this paper does not take a stand with regards to the polarized digital native debate (Bennett et al. 2008), it does acknowledge that students attending higher education are indeed engaged with technology and bring to their studies a wealth of experience of using technologies in their everyday lives, whether they are generationally unique digital natives or not. Furthermore, the hypothesis that guides this work—with support from Kress (2003), Jewitt (2008) and Selander (2016)—is based on the idea that some of the experiences that students bring to their studies are associated with the use of multimedia and multiple semiotic resources for meaning-making and learning.
It is against this backdrop that this paper will investigate the following research question: how do university students, in a computer and systems science department, during self-study design their learning using digital learning material and different modes and media? This paper reports on a mixed-method study that was conducted at the department of Computer and Systems Sciences at Stockholm University, Sweden during 2015. In this paper, we will describe how university students exploit multimodal literacy and the affordances of different modes and media in order to support construction of knowledge during self-studies. As such the findings of this paper calls for a pedagogical shift that take departure from a design-oriented, multimodal understanding of learning (Kress and Selander 2012).
2 A Multimodal Design Theoretical Perspective on Learning
This study draws on multimodal perspectives of learning and meaning making as outlined by Selander (2016), Selander and Kress (2010), Jewitt (2008) and Kress (2003). Multimodal perspectives on learning and meaning making emerged as a response to new media practices of learners (Kress 2003) and rest on the basic assumption that meaning are made through many representational and communicational resources (semiotic resources), of which textual language is only one (Jewitt 2008). From a multimodal perspective, modes, such as gestures, audio, video and images are semiotic resources that have different affordances and potentials for making meaning. That is, the different modes have unique capacities to express and represent things based on their material, physical and environmental constitutions (Jewitt 2008). For instance, an image on a two-dimensional surface offers different potentials for the representation of meaning than the affordance of speech and sound. As explained by Jewitt (2008), the two different modes are also governed by different logical structures; the sounds of speech occur in time and are thus governed by a temporal logic, while the affordances of images can be understood as governed by the logic of space and simultaneity. The two different underlying logics produces unique possibilities and constraints for representing and communicating meaning.
The modes and media are in the multimodal perspective crucial aspects of learning and knowledge construction (Kress 2003). As explained by Jewitt (2008), the modes and “the ways in which something is represented shape both what is to be learned, that is the curriculum content, and how it is be learned” (p. 241). Thus, the multimodal lens has significant implications for how learning is conceived. To elaborate, from a multimodal perspective, the emergence of digital technologies and new media has created new conditions for learning, as new semiotic resources have been made available for consuming and producing knowledge representations. The alternative semiotic resources, such as audio, digital images and video, complements and at times replaces the historically dominating semiotic resource of the printed text, and consequently expands what is possible to express and represent, and extends the repertoire of tools learners can make use of in order to learn. As emphasized by many proponents of the multimodal approach, in the new technological media landscape, learners are more active producers of knowledge that design their learning by selecting among and by utilizing multiple semiotic resources (Selander 2016; Jewitt 2008; Kress 2003).
The concept of design within the multimodal approach has been elaborated by several. For instance, Cope and Kalantzis (2009) differentiate three aspects of design, namely: (1) Available designs (found representational forms); (2) the Designing one does when making meaning and appropriating and transforming available designs; and (3) The Redesigned (how through the act of Designing, the world and the person is transformed). From this perspective, designing is the act of doing something with available designs such as communicating to others through different modes or representing the world to oneself (ibid.). The learner is described as the “meaning-maker-as-designer”.
A similar account of design in relation to multimodality, conceptualized as a multimodal design theoretical perspective, is proposed by Selander and Kress (2010) and Kress and Selander (2012). In their view, a distinction is made between design for learning and design in learning. The first, design for learning, represents the didactical design of teachers that is done when planning and arranging learning activities and environments, while the second, design for learning, represents the choices the learner does in consuming, transforming and constructing representations during the learning process.
In this study, a multimodal design theoretical perspective is adopted that views learners as active designers of knowledge and that acknowledge that learning and meaning making—in the current technological media landscape—can and many times are taking place trough the use of different modes and in a multimodal way. Supported by the distinction of multimodal design of learning made by Cope and Kalantzis (2009) and Kress and Selander (2012), in this study, we investigate how higher education students design their learning during self-study: (1) by using available designs and representations in different modes; and (2) by designing and producing own learning material and learning representations in different modes.
3 Methodology
The study presented in this paper was conducted at the department of Computer and Systems Sciences at Stockholm University, Sweden during 2015. The department currently hosts approximately 5400 students, who attend eight Bachelor’s and nine Master’s programs within computer science. This research used a mixed-method approach consisting of data collection in form of a questionnaire and face-to-face interviews that examined what kind of multimodal study practices the students perform and value. The mixed method approach allowed us to capture and synthesize data from multiple sources in order to gain in-depth and comprehensive understanding. The questionnaire was developed and offered to students at the department through a web tool. The selection of students for the questionnaire was based on one criteria: the students should be active and have completed courses at the department. Among the participants, 14 students were randomly selected to take part in face-to-face interviews.
3.1 Participants
In total, 505 students participated in the questionnaire. The students, 235 females and 270 males, ranged in age from 19 to 66 years, with a mean age of 28.51 years (SD 7.46). In terms of levels of study, 71.1% of the students attended a Bachelor’s program, 19.6% a Master’s program, and 9.3% studied independent courses. On average, the students had studied at university level for 4.11 years (SD 2.89). Table 1 details the students’ characteristics.
Table 1
Students’ characteristics
Background questions |
n
| % |
M
| SD |
|---|---|---|---|---|
Age | 505 | 28.51 | 7.46 | |
Gender | ||||
Women | 235 | 46.5 | ||
Men | 270 | 53.5 | ||
Study level | ||||
Bachelor’s program | 359 | 71.1 | ||
Master’s program | 99 | 19.6 | ||
Independent courses | 47 | 9.3 | ||
Years in higher education | 494 | 4.11 | 2.89 | |
Average grade during study | 505 | 3.14 | 0.82 |
The average age of the 14 students’ chosen for face-to-face interviews, 6 females and 8 males, was 27.81 years (SD 5.43). In average, these students had studied on university level for 4.92 years (SD 2.31).
3.2 Data Sources and Analysis
3.2.1 The Questionnaire
A questionnaire was developed consisting of three sections with 59 items to measure the students’ multimodal learning practices during self-studies, as follows:
-
Section 1 consisted of 12 demographic and background items.
-
Section 2 consisted of 24 items examining students’ productions of multimodal digital material during self-studies with a particular focus on the four modes, namely: text, image, audio, and video (see Table 2).Table 2Production of digital learning material using different modesProduction of digital learning materialn%How common is the practice for you?*PurposeDo you write digital texts for learning purposes?Summarizing the lectures, course literature, video material, and of own and others notes. Material made more accessible, editable, and sharable for collaborationYes37073.3M = 3.05, SD 0.77No13526.7Do you take pictures for learning purposes?Taking pictures of lecture slides, drawings and notes made by teachers, own and other students notes, collaborative brainstorming sketches, interesting pages in articles and books, and in order to visualize own thoughtsYes28756.8M = 3.01, SD 0.73No21843.2Do you audio record yourself or others for learning purposes?Recordings of lectures, their own reasoning when studying, summaries, and other students’ explanations and group discussionsYes6312.5M = 2.43, SD 0.99No44287.5Do you video record yourself or others for learning purposes?Recordings of themselves during preparation for presentationsYes193.8M = 2.50, SD 1.00No48696.2
-
Section 3 consisted of 23 items examining the students’ use of pre-existing multimodal digital material during self-studies with a particular focus on the four modes, namely: text, image, audio, and video (see Table 4).
An exploratory factor analysis with principal component extraction was performed in an attempt to refine the instrument. After factor analysis, nine items that did not load on any factors or highly cross-loaded on multiple factors were removed. Accordingly, the instrument used for the final analysis consisted of 50 items.
The questionnaire was developed and administered through a web tool. With regard to the analysis of the questionnaire data, descriptive statistics were calculated. Pearson correlation analyses were performed to examine the linear relationships between the different multimodal study practices. Independent sample t tests and Anova-tests were performed to examine statistical differences between groups of students with regard to the multimodal study practices.
3.2.2 The Interviews
The interviews were semi-structured, consisted of three parts and contained:
1.
Demographic questions (age, years in higher education, etc.).
2.
Questions about how the students using different semiotic modes constructed digital material during self-studies.
3.
Questions about how the students used pre-existing multimodal digital material during self-studies.
The interviews were fully transcribed and coded according to procedures of thematic analysis (Braun and Clarke 2006). In this particular paper, a selection of the identified qualitative codes was used to support, clarify, and elaborate the presented quantitative findings.
4 Results
This section will present the analysis of the students’ modal preferences during self-study and it is divided into two sections. The first section presents the results of students’ productions of digital learning material through different modes. The second section presents the results of students’ use of pre-existing multimodal digital learning material.
4.1 Production of Digital Learning Materials Using Different Digital Modes
In the first section of items, the students were asked about how they construct externalized digital artifacts and materials in different modes in order to support learning during self-study. Table 2 gives an overview of the students’ modal preferences when constructing learning material.
As can be seen from Table 2, the most popular activity was writing digital texts, followed by taking pictures, audio recordings, and video recordings. The listed purposes for doing these activities stem from the thematic analysis of the qualitative interviews.
4.1.1 Writing Digital Texts
Not surprisingly, the production of digital texts was the activity that most of the students (73.3%) declared to have practiced during self-study in order to learn the course material. This type of production activity was also the most used during self-study among the students who wrote digital texts for learning purposes (M = 3.05, SD 0.77). The qualitative analysis of the interview data showed that the students mainly wrote summaries of lecture material, course literature, video material, and of own and other students notes. Several reasons for preferring digital texts were reported, for instance: a digital text is more accessible; facilitates sharing and collaboration; and, a digital text can be continuously edited. Interviewee 1 expressed the collaborative aspect in the following way:
A good example that has worked for me in courses. When you have some form of pre-exam questions handed out by the teacher. We answer them one by one in a shared Google Document. And then you can correct each other and help each other out to answer the questions (Interviewee 1).
4.1.2 Taking Pictures
The analysis reveals that a large portion of the students (56.8%) took pictures for learning purposes during self-study and that this is almost as commonly practiced as constructing digital texts (M = 3.01, M = 0.73). The interview data indicates that the students take pictures (mainly with mobile phones) of, for instance: lecture slides, drawings and notes made by teachers on whiteboards; their own and other students’ notes; collaborative brainstorming sketches; and, interesting pages in articles and books. The main reasons that were given for taking pictures were that digital pictures can be stored and are accessible, shareable, and editable. Interviewee 2 expressed these three reasons quite clearly:
Well I use my smartphone and either take photos of drawings that the lecturer made or during group works if we take notes or make drawings on the board. I also take pictures on drawings, mind maps or texts that I have written myself and upload them to my Google drive. Then I can access them anywhere and anytime and edit them as much as I want. And I can share them with my friends (Interviewee 2).
4.1.3 Making Audio Recordings
The practice of audio recording for learning purposes was less unusual than expected. Among the students answering the questionnaire, 12.5% (n = 63) reported that they have audio recorded themselves or others during self-study and that it was not an uncommon practice (M = 2.43, SD 0.99). The qualitative analysis of the interviews showed that students typically make audio recordings of: lectures; their own reasoning when studying for a subject; summaries; and other students’ explanations and group discussions. Several reasons for making audio recordings were identified in the qualitative data. For example, the students mentioned that they helped to capture relevant discussions in speech for flexible recap, analysis and reflection, which can be demonstrated by the following excerpts:
Audio recordings allows me to study more efficiently while I am doing something else, helps me to multi-task. For instance, when I go to gym and stuff, I don’t want to listen to music but I want to learn, it’s like I have really little time to squeeze between everything so I do try to record a lot (Interviewee 2).
I do that when I can’t keep up with someone and really need to get a hold of all the information being presented. I record teachers during lecture sometimes. And sometimes I record our group discussions during exam preparations. And then I listen to that in my own pace and understand the information better (Interviewee 3).
4.1.4 Making Video Recordings
The least reported construction practice in the questionnaire was video recordings. Fewer students (n = 19) reported that they have video recorded themselves or others for learning purposes during self-studies: a practice that was not that untypical (M = 2.50, SD 1.00). Detailed qualitative data could not be retrieved regarding what the students made video recordings of or the reasons for choosing video as the medium. Nevertheless, Interviewee 4 expressed the view that:
Well, it has happened several times. I record myself when I prepare presentations. The recordings help me see what I do good and what I need to improve (Interviewee 4).
This quote indicates one use, namely capturing presentation performance. The reason was to assess their strengths and weaknesses, and for regulating and improving behavior.
4.1.5 The Relations Between the Modal Practices and Differences Between Groups of Students
Regarding the relations between the practiced production activities, a significant correlation was found between the writing of digital texts and taking pictures (r = 0.21, p < 0.01). The students that write digital texts during self-study to support their learning also tend to take pictures, which emphasizes a preference for digital productions of learning material as the common underlying factor. With regard to the practice of audio recording themselves or others for learning purposes in relation to the other modal production practices, no significant correlation was found. The practice of video recording for learning purposes was weakly correlated with taking pictures (r = 0.17, p < 0.01) and modestly correlated to the practice of audio recording (p = 0.36, r < 0.01). Table 3 gives an overview of the correlations between the variables.
Table 3
Correlations between variables
Variable | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
1. I write digital texts | 1.00 | |||
2. I take pictures | 0.21** | 1.00 | ||
3. I audio record myself or others | 0.21 | 0.15 | 1.00 | |
4. I video record myself or others | 0.10 | 0.17** | 0.36** | 1.00 |
Multiple independent t tests were conducted to examine the statistical differences between Bachelor’s and Master’s students in terms of how common a modal practice is for them. A one-way Anova test was conducted to examine the differences between student groups with different average study grades. No differences were found between the groups with regard to writing digital texts, taking pictures, making audio recordings and video recordings. However, the number of years in higher education was correlated to the practice of taking pictures (r = 0.20, p < 0.01).
4.2 The Students Use of Pre-existing Digital Learning Material
The students were also asked about how they use pre-existing digital learning materials in different modes in order to support learning during self-studies. Table 4 gives an overview of the modal preferences of the students when using pre-existing learning material.
Table 4
Use of pre-existing multimodal learning material
Use of pre-existing multimodal learning material |
n
| % | Scope of practice* | Purpose |
|---|---|---|---|---|
Do you use pre-existing video for learning purposes? | Broaden perspectives, learn in their own pace, gain deeper understanding, get more motivated, assist reading of literature, and becoming more mobile | |||
Yes | 425 | 84.2 | M = 2.84, SD 0.90 | |
No | 80 | 15.8 | ||
Do you use pre-existing texts for learning purposes? | Gained understanding, in order to share, mobility, and accessibility | |||
Yes | 417 | 82.6 | M = 3.24, SD 0.77 | |
No | 88 | 17.4 | ||
Do you use pre-existing images for learning purposes? | Support understanding and comprehension, and accessibility | |||
Yes | 245 | 48.5 | M = 2.56, SD 0.85 | |
No | 260 | 51.5 | ||
Do you use pre-existing audio for learning purposes? | Support understanding and comprehension, and mobility | |||
Yes | 59 | 11.7 | M = 2.43, SD 0.92 | |
No | 446 | 88.3 | ||
4.2.1 Using Pre-existing Video Material
Rather surprisingly, the practice of using pre-existing video material was the most reported practice among the students and the second most common practice (M = 2.84, SD 0.90). In other words, more students reported that they have used pre-existing video material to support their learning (84.2%) than students reporting on the use of digital texts (82.6%). Qualitative analysis of the interview data showed that the students mainly used pre-existing video material to broaden perspectives, learn at their own pace, gain deeper understanding, get more motivated, and assist reading of literature. In the following quote, one of the students emphasized their motivation and the change of perspectives:
Meanwhile, another student, Interviewee 6, reported that recorded lectures gave her opportunities to reflect and learn at her own pace:It is easier to understand things in video format and books are sometimes boring. When I do not understand what the book is talking about, or my teacher’s explanations, I find video lectures on the subject. I follow Youtube and I’ve subscribe to some scientific channels. In that way I get to see the subject from another perspective or explained differently or better (Interviewee 5).
With video you can pause and reflect, and then write text down in a more reflective manner, in a pace that is good for you. You cannot do that during a live lecture. So I use the real lectures to get the main picture and then I use the recordings to get more into detail (Interviewee 6).
4.2.2 Using Pre-existing Digital Texts
A large majority of the students (82.6%) used pre-existing digital texts from the Internet to support their learning. The use of pre-existing digital texts from the Internet was the most repeated practice among the students (M = 3.24, SD 0.77). The qualitative analysis revealed that students used pre-existing digital texts because they offered flexibility and helped them to gain understanding, availability, accessibility, shareability, and mobility. Interviewee 4 underlined several of these advantages:
I like digital texts better because you have it on your computer so you don’t need to carry it with you if you go to school, not having to bring the physical thing with you. And most likely I can read it easily from everywhere at any time I want, like home, school or in the bus using my phone. It’s also easy to get and it’s also an easy way to learn something fast, especially if it’s just something that the lecturer just mentions and you just want to get some more information on it, then it’s a good way to just skim it and get a good overview of the area (Interviewee 4).
4.2.3 Using Pre-existing Digital Images
Quite a lot of the students (48.5%) had also used pre-existing images from the Internet to support their learning. Among these students, the practice of using digital images was quite frequently performed (M = 2.56, SD 0.85), albeit not as common as the use of video or digital texts. The main reason for using digital images was the accessibility of the images and because the visual representations can support understanding and comprehension. For instance, the following student reported that:
If I don’t understand some term or other word or important terms it is easy to Google images that can help me grasp them, and understand the terms (Interviewee 7).
4.2.4 Using Pre-Existing Audio
The use of pre-existing audio to support learning was quite uncommon and was experienced by only 11.7% of the students. Among these students, this practice was also the less repeated practice (M = 2.43, SD 0.92). In the interviews, one student expressed that he is an auditory learner, which was one of the reasons for using audio. He reported that audio supported flexible learning, as expressed by the following:
Well I actually listen to YouTube video clips on my phone. It is a video, but I listen to the audio in the video clip because I’m at the gym and cannot look at the video. And I learn better when I listen or look than reading (Interviewee 3).
4.2.5 The Relations Between the Modal Practices and Differences Between Groups of Students
The relations between the practices of using pre-existing material in different modes showed that significant inter-correlations could be found between all of the variables except between the use of texts and audio (r = − 0.18, p > 0.05) (see Table 5). Among the students relying on audio, a rejection of the written modality is indicated and a preference was reported for video (r = 0.31, p < 0.05) and images (r = 0.38, p < 0.05). In fact, the correlations between all variables and the variable “use of digital texts” is the weakest between video and text (r = 0.19, p < 0.01), images and text (r = 0.22, p < 0.01). Put differently, those that rely on video, audio and images tend to rely less on the written modality.
Table 5
Correlations between variables
Variable | 1 | 2 | 3 | 4 |
|---|---|---|---|---|
1. Use of pre-existing video for learning purposes | 1.00 | |||
2. Use of pre-existing texts for learning purposes | 0.19** | 1.00 | ||
3. Use of pre-existing images for learning purposes | 0.24** | 0.22** | 1.00 | |
4. Use of pre-existing audio for learning purposes | 0.31* | − 0.18 | 0.38* | 1.00 |
Multiple independent t tests were performed to examine the differences between Bachelor’s and Master’s students, and between men and women in terms of how common a modal practice involving pre-existing material is for them. No differences between Bachelor’s and Master’s students were revealed with regards to use of pre-existing video, audio, and images. However, the Master’s students (M = 3.48, SD 0.66) significantly found more support in pre-existing digital texts on the Internet during self-study as compared to Bachelor’s students (M = 3.18, SD 0.78), t(412) = − 3.5, p < 0.05. No differences were noted between the genders, except that men (M = 2.97, SD 0.90) tended to find support in pre-existing video more than women (M = 2.67, SD 0.89), t(418) = − 3.54, p < 0.01. Although a one-way Anova test was conducted to examine the differences between student groups with different average study grades, no statistical difference was revealed. However, the number of years in higher education was positive correlated to the practices of using pre-existing images (r = 0.22, p < 0.01) and pre-existing digital texts (r = 0.12, p < 0.05).
5 Discussion
By looking at the results obtained, one can arrive at several conclusions. One of the general conclusions of this study corroborates Kress (2003) and Jewitt (2008), who claim that technology in a revolutionary way have changed how knowledge is communicated and represented. Selander and Kress (2010) can also be corroborated, who emphasize that meaning making and knowledge building is increasingly taking place in a multimodal way. The findings of this study show that the way university students make meaning and build knowledge during self-study have changed; that technology make available other semiotic resources for them than the text thus transforming monomodal learning practices into multimodal learning practices. What could be seen is that the historical dominating text as a semiotic resource is no longer of paramount importance, neither when consuming pre-existing digital learning material or when producing digital learning material in knowledge building activities. For instance, in terms of knowledge acquisition and consumption of existing digital learning material, video as a semiotic resource was more popular and more frequently used among the students than texts. In addition, the use of audio and images were incorporated in many students’ learning habits. In terms of the students’ own production of digital learning material and supportive representations of various kinds during self-study, it can be noted that the activity of taking pictures was almost as popular as the production of digital texts. A reasonable portion of the students also produced video and audio recordings to support their learning of course content, regularly supported by mobile technology.
Thus, students’ construction and consumption of learning material is to large extent taking place in a multimodal way. The reason for that, one could argue with support of Kress (2010), is that “different modes offer different potentials for meaning making” (p. 79), and that students utilize the affordances of different modes (audio, video, pictures, etc.) because it helps them better construct knowledge than when exclusively bounded to the mode of text. By being able to do different semiotic work with different modes, enhanced opportunities to make meaning out of course content and course activities are offered. An example of this can be seen in the activity of taking pictures of lecture notes and the digitalization of sketches made by students of written reflections/thoughts. What is achieved by such a digital materialization, as Sörensen (2009) or Säljö (2010) would put it, is enhanced accessibility, shareability, analysability, reusability, and mobility—benefits and affordances that the interviewed students’ also highlight. Another concrete example of enhanced learning opportunities, and individualization of learning, can be found in the students use of video for knowledge acquisition or knowledge representation, which allow for learning at their own pace (pause, repeat), flexible/mobile learning on the go, and broadening of perspectives (access to many alternative perspectives).
Hence, in a sense, the findings portray the university students of today as learners that in a mobile and flexible way use the affordances of different technologies, particularly portable mobile technologies, in order to construct knowledge through different semiotic modes when they are to create meaning out of course related content and activities during self-studies. In that way students are active designers of meaning and learning (Cope and Kalantzis 2009) that draw purposively and selectively from a vast breadth of semiotic resources in order to construct knowledge. As Cope and Kalantzis (2009) claim, learners have different modal preferences for representation and are more comfortable in one mode than another. Consequently, by being able to select modes tailored to the needs and preferences of learners more students might be included in learning. More importantly, the move from the paper to the screen (for instance mobile technology) give students access to new semiotic resources for representing and constructing knowledge, that extends what can be done with only paper and pen as demonstrated in the conducted interviews. Thus, opportunities are obtained for enhanced and enriched learning processes.
Nevertheless, the fact that modes have different affordances and constraints for shaping knowledge, that students can chose among more modes than before for construction and representation of knowledge, and that they can combine different modes for multimodal learning, implicate that the process of knowledge construction and learning has become more complex—characterized by design, diversity and multiplicity (Jewitt 2008). And as a consequence, whether teaching practices take the multimodal learning practices of students into account and align to them or not, the act of teaching becomes more complex and will require the development of multimodal literacies (Cope and Kalantzis 2009; Sanders and Albers 2010; Goodfellow 2011).
The findings of this study point at that students commonly use different modes in their learning practices. Based on these findings, and supported by the observations of Jewitt (2003, 2008), Walsh (2010), and Sanders and Albers (2010), there are indications that many of the students participating in this particular study—through formal and informal activities—have developed multimodal literacies in different degrees in order to be able to harness the potentials and address the complexity of multimodal learning. There is little reason to believe that skills and literacies have not been developed for multimodal learning considering that the findings demonstrate that students repeatedly and commonly resort to consuming and constructing learning material in different modes when attempting to make meaning out of course content.
However, although it has been indicated in the interviews conducted in this study that learning is more effective and meaningful during self-study when students can choose modes of knowledge representation and construction, more research is needed that in detail scrutinize the relation between, on the one hand, affordances of different modes, multimodal literacies and practices, and on the other hand, learning and performance. Indeed, this study is limited in that it does not delve into depth with regards to the question of how students are using different modes in order to make meaning out of course work during self-studies, how knowledge is reshaped and how learning and performance is affected. The findings of this study should also be viewed in light of the other limitations of this study. One such limitation is that all participating students study computer science, which can not be considered as entirely representative for a larger population of students with various educational orientations, albeit digitalization has influenced large portion of societies. Future studies with a broader sample with higher variance in education topic is recommended.
As final remarks, these results prompt teachers to extend their pedagogical repertoires and calls for a development of a new pedagogical approach that may rest upon what Kress and Selander (2012) describe as a design-oriented, multimodal understanding of learning. This approach should, on the one hand, recognize that students are active designers of meaning with multimodal literacies and learning skills, in order to deliberatively build teaching upon and support the development of such literacies and skills. On the other hand, this pedagogical approach should also reconsider historically dominating monomodal assessment practices and acknowledge the need for a new culture of recognition (Kress and Selander 2012).
Certainly, one can ask whether higher education should adapt and align by consciously supporting the development of students’ multimodal literacy and learning skills, and by exchanging its current monomodal view of assessment with multimodal assessment practices. The answer might be straightforward. If higher education students have developed multimodal learning practices, as the students of the study sample seem to have done, then it is entirely reasonable that teaching in the form of instruction, design of learning environments, and assessment should also take these practices into account and support them in the best possible way. A mismatch between the students’ learning practices and the teaching practices of higher education should not be considered to be a desirable outcome.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.