Introduction

Today, Swedish students use mainly digital tools to search for information and to write texts in social science and national language classes (Board of Education 2016). An increased use of digital tools and resources for learning has entailed an extension of opportunities and challenges for student's writing repertoires. According to this digitized development, there are expectations of new and creative forms of learning and new ways of representing knowledge in any subject. Increased access to digital tools and resources is also challenging the students’ choices of different modes for representing experiences and learning (Åkerfeld 2014). Different digital tools, websites and software applications, offer a wide range of modes (for example writing, images, speech, and moving images) for meaning making and communication. Multimodality may be seen as a field of application (Kress and van Leeuwen 2001) where multiple modes provide potentials of engagement and meaning making in the world. All modes are seen as equal communicational forms that students use in their repertoire of making meaning and knowledge visible in communication. However, all modes are not always recognized and assessed by teachers in schools (Kress and Selander 2012) because of the modes different communicative work. A written text has by tradition a higher status in assessment than an image which is often only recognized as a decoration to the written text (Jewitt 2009; Oldakowski 2014).

Traditional print-based activities have also historically dominated Swedish writing and assessment practices. Learning to write has been conducted exclusively by means of paper and pen. Even the notion of writing and literacy education has hitherto been focused on concepts of language proficiency and on knowledge and skills conducted in the form of written text. In relation to the ongoing digitalization occurring in schools, a re-conceptualization of literacy has been developed (New London Group 2000) and a new, wider understanding of text has emerged. The meaning of text has also changed and expanded to include modes of meaning making other than just printed text. A text that has more than one mode is considered to be multimodal (Kress and van Leeuwen 2001; Danielsson 2011; Selander and Danielsson 2014) and it has taken on a wider meaning by including multimodalities. A text may, on one hand, be understood as a resource in any mode which carries meaning, and on the other hand, a resource that orchestrates various modes (for example images, writing, and speech) of meaning making (Kress and van Leeuwen 2006; Selander and Danielsson 2014). To large extent, students are more than ever taking part in all kinds of contemporary text-making (multiliteracies) activities (New London Group 2000), which can be understood as a wide range of literacy practices (Barton 2007) where students engage with multimodal texts (Jewitt 2008; Hung, Chiu and Yeh 2013; Cope and Kalantzis 2015, Bezemer and Kress 2008). Within such literacy practices, students create texts using different digital tools and modes to represent experiences and knowledge (Jewitt 2008). Given these shifts, there is a need to explore whether the ways of viewing writing and recognizing knowledge in multimodal texts have changed in schools. In light of these ongoing changes in the digital and literacy landscapes in schools, this article aims to explore student's use of multimodal texts and how students and their teachers perceive and recognize the multimodal texts produced in three different literacy practices such as log books, written reports, and oral presentations.

The article poses three research questions:

  1. (i)

    how do students design their interest and understanding of the task using different digital tools and modes of representation?

  2. (ii)

    how do teachers and students perceive the design of multimodal texts?

  3. (iii)

    what do the teachers and students recognize as valuable representations of knowledge?

Literature review

The research fields of multimodality, (Bezemer and Kress 2016; Jewitt and Kress 2003; New London Group 2000) and New Literacy Studies (Blåsjö 2010; Aamotsbakken 2008; Cope and Kalantzis 2015) focus on text and literacy in the broader sense. These fields place text-making or writing in a multimodal framework, and emphasize the importance of focusing on both the practice in which the text-making occurs and on the text design process which takes place through a range of media and modes. They view text and even writing as multimodal, there linguistic, visual, auditory elements can be including in the composition (Edwards et al. 2009; Edwards-Groves 2011; New London Group 2000; Hung et al. 2013; Cope and Kalantzis 2015). Ivanič (2004) proposes also that people who discuss texts and writing are often doing so in discursively hybrid manner. People are constantly under the influence of socially available discourses, and these discourses are continuously transformed and heterogeneously used. Six different discourses, skills discourse, creativity discourse, process discourse, genre discourse, social practices discourse and sociopolitical discourse, are identified as influencing beliefs surrounding writing and learning to write.

In a design process of multimodal text, the students give shape to interest, purposes and intentions by using different media, tools and modes (cf Kress and Bezemer 2008; Selander and Kress 2010). They use, for example, modes (printed text, images and speech) in order to express their understanding of a subject in an educational setting. Printed written text, image, speech, moving images, colors, music, gestures are all examples of modes which people use in composing a multimodal text. These different modes that the students use to design multimodal texts shape the student's representation of engagement differently. The notion of representation is an expression of how students choose to render visible their understanding of different content and knowledge. Representations of knowledge are always situational and are recognized and assessed in a social and cultural domain. The recognitions of these representations are also influenced by temporal and hegemonic cultures. Different representations have different semiotic potentials and constraints which lead to the result that different aspects of the content will be highlighted, recognized and assessed (Kress and Selander 2012).

There are several research studies that highlight how texts are influenced by new digital media (MacArthur 2006; Gee 2007; Öman and Sofkova 2015; Sofkova Hashemi 2017). These studies show that the design of text is becoming a more complex, multimodal means of communication, representing a shift from a traditional way of using printed text to more multimodal use of texts. The use of digital tools may also challenge students’ perception of texts and their idea and recognition of multimodal text productions. Kjällander (2011) claim that students are able to design texts digitally, by combining different modes in more creative ways and they are also able to go beyond what is taught by teachers in their own designing process. Even educational research (Blåsjö 2010; Öman 2015) has shed light on novel changes in the text cultures of schools and has called for a pedagogy of multiliteracies (Cope and Kalantzis 2015). Many studies have emphasized the importance of understanding the challenges of using digital media and their effects on students’ multimodal writing repertoires (Jewitt and Kress 2003; Axelsson and Danielsson 2012; Selander and Danielsson 2014; Björkvall 2009). The use of different modes of text production in combination, such as use of images, animations, speech, music and written texts, brings forward an understanding of new approaches of writing (Edwards-Groves 2011). Other previous studies (Oldakowskis 2014; Åkerfeld 2014) have also highlighted the importance of multimodal learning and the recognition of multimodal texts in educational settings. Oldakowski (2014) points out that often in education, mostly traditional text is evaluated and assessed. Svärdemo Åberg et al. (2013) show that digital writing technologies have an impact on pupils’ writing process and that digital tools and different use of modes shape communication and even in some extent the possibilities of representing knowledge in a given situation. McGrail and Behizadeh (2015) pose also the teacher's need to developing their own framework which is based on multimodal composing so that students can understand the unique convensions for creating multimodal texts. Even writing practices in the academy and in higher education are changing, and scholars pose the importance of going beyond writing and including multimodal compositions and other text-making practices (Björkvall 2016; Hernwall et al. 2016; Huang and Archer 2017). According to these ongoing changes in the digital landscape, it is important to be aware of the potentials and the limitations in new and emerging literacy practices. Based on the review of previous literature there is a lack of research that focus on design processes and assessment practices of multimodal texts in schools. Therefore, we see our study as a contribution to the education research field.

Methodology

The empirical context

The context of this study regards what students and teachers do and experience within six different project assignments in two upper secondary schools in Sweden. These two secondary schools where both selected with an eye to presentations of websites where the use of ICTs were highlighted and emphasized. School A is one independent school with a specific one-to-one profile. This school provided one laptop computer to each of the students at the school. School B is a municipal school with no specific ICT profile. Computers were mainly available in specific computer rooms which the students could visit any time during the school day. However, the students in school B were allowed to bring their own computer to the teaching classes at the school.

The project assignments were related to a mandatory course at the upper secondary high school. The students were in their final year and the project assignment was one of the core courses before graduation from upper secondary high school. The project assignment had specific learning outcomes which were defined by a syllabus, and accordingly, each student should be involved in a process of conducting investigations into developing new content knowledge on a specific subject matter. A project assignment could result in, for example, an object, a service or an answer to a theoretical problem. The project assignments could also be varied in form as well as content. A central aspect stated in the course plan was that the process was equally important as the final product. During the project assignment, the students were obligated to document their process in a log book and at the end of the course also write a report about the whole project assignment. At the end of the course, the project assignment also had to be presented orally for the class or the school. At the beginning of the course, the teachers, at both schools, gathered the students for an introduction. The learning objectives and grading criteria were presented and the organization of supervision of the different texts were explained. Also, at the beginning of the course, all teachers were obliged to offer several lecturers about the scientific writing process and explanations of what a report should contain in terms of content and form. All teachers ought also to explain their demands concerning the disposition structures of how to write a scientific report. The teachers and students involved in these project assignments met over two semesters (from fall to spring). During the course and the whole time the students were working with the project assignments, they got regular supervision by the teachers.

Selection of project assignments

The project assignments were selected to be suitable for the purpose of the study, by a purposive sampling (Cohen et al. 2011). In dialog with the teachers we handpicked the project assignments on the basis of typical cases, i.e., representativeness (cf Teddlie and Yu 2007) and the criteria were: (a) diversity in subject of inquiry both in social sciences and natural sciences; (b) diversity of available digital tools for learning (each student had access to a laptop computer and the school's learning management system and Internet); (c) diversity of literacy practices, all students needed to produce a log books, report, and an oral presentation. In school A, two project assignments (Anti-bulling plan and Life coaching) were selected and they were conducted by totally six students within the subjects of social science. One teacher supervised both these projects (Table 1).

Table 1 School A, project assignments, subjects, students, digital tools
Full size table

In school B, four project assignments (Social media and carbon dioxide emissions, Antibiotics processed from fungi, Food waste and Preservation) were selected and they were conducted by totally seven students within the subjects of natural science. Two teachers supervised two project assignments each (Table 2).

Table 2 School B, project assignments, subjects, students, digital tools
Full size table

Data collection methods

A ethnographic approach (Murchison 2010) was used as a research strategy that allowed us to explore the context of social practice and to examine how the students, in each project assignment, designed their texts in three different literacy practices such as log books, written reports and oral presentations. The data was collected through a combination of field observation, video documentation of classroom interactions and observations in the learning management systems and other computer-mediated communication, i.e. in Facebook (Kozinets 2010) and by interviews. To be able to get a deeper understanding of the social and educational contexts in the two schools, we conducted a number of participant observations in classes where the students received supervision from the teachers. During the whole course, two semesters, the students met their supervisors (teachers) several times. We took field notes of the supervision activities. The final oral presentations, when the students were presented with their final results, were recorded by video camera. We also made continuously observations of activities within Learning Management System (LMS) and in a private Facebook group. We collected documents of the student's and the teachers’ communication and the students’ text productions in log books, written reports and oral presentations (Murchison 2010). External syllabus and internal policy documents were also collected and scrutinized. In the end of the course we conducted interviews with both teachers and students. Interviews were conducted with totally thirteen students and three teachers. The interviews with the students were done in groups and individually with the teachers. The interviews were carried out during school hours and each interview lasted approximately forty-five to sixty minutes. We organized the interviews in relation to a schedule, a list of interview questions which covered topics about supervision, writing in log books, written reports, oral presentation and recognition of knowledge in all the three literacy practices. All interview data were recorded using mobile phone and video camera. Altogether, estimated data were 320 min of field observation and video documentations of different activities (i.e., supervision in classrooms, collaboration, and oral presentations in classrooms), nine log books, six written reports, and sixteen interviews (thirteen students and three teachers: one teacher of social science and two natural sciences teachers). In this study, we used the student's multimodal text productions (log books, written reports and oral presentations) and the interviews for the analysis.

Data analysis

To address the first research question, data were collected through the students’ text documentations within the six different project assignments. Each student produced a log book (some students wrote in pairs), a written report, and an oral presentation (accompanied by a PowerPoint presentation). In order to explore which texts were designed in the different project assignment, we carried out an overall comprehensive analysis of corpus data to examine which digital tool (media) and modes were used for representations of knowledge, and in which variations they occurred. The analysis of data focused on the three literacy practices in the project assignment—log book, written report and oral presentation (c.f. Kress and van Leeuwen 2006). First, the analyses were focused on the form of these texts; the overall structure and setting, the intrigue and sequencing, thematic orientation, and the affordances of the different resources (Selander and Danielsson 2014). Furthermore, the analysis was focused on how the subject content was framed within the different media and tools, on what modes (i.e., written text, images or speech) were used and how they worked together. In the analysis, we were especially interested in how these different multimodal texts were related to a genre of writing. We looked at personal narratives or academic conceptual representations and we explored which metaphors were used in these texts. Furthermore, the analysis covered how these representations in the different texts addressed contact to the recipients (teachers or other students). This analysis also involved how the students chose to represent themselves and their attitudes to the subject matter. Finally, we organized the students’ multimodal text productions to rubrics of design. We used the rubrics linguistic design, visual design and auditory design (New London Group 2000; cf Hung et al. 2013; cf Sofkova Hashemi 2017) for describing and summarizing a complex and dynamic engagement with modes of communication in the log books, the written reports, and the oral presentations. We considered which digital tools were used and how texts in the log books, written reports, and oral presentation were designed.

Linguistic design What linguistic design elements (e.g., written style and structure, content and conceptual usage, spelling, grammar, coherence, narration) were represented in the multimodal texts?

Visual design What visual design elements (e.g., frame, images, tables, photos, layout, colors) were represented in the multimodal texts?

Auditory design What auditory design elements (e.g., sound effects, music and oral style and structure, content and conceptual usage, coherence, narration) were represented in the multimodal texts?

In order to handle the second and third research questions, we conducted interviews with teachers and students, both individually and in groups. The focus of the interviews was towards the student's and teachers’ perceptions of the whole learning process as part of the project assignment, their views of the supervision, writing and design process in log books, written reports and oral presentations, and the recognition of multimodal texts according to scoring rubrics. All interviews were recorded, transcribed, and digitally annotated for analysis. The analysis of the interviews was carried out in several steps. The first step involved an overview of data for identifying central perceptions of the learning process in the project assignment, the literacy practices, the text-making and assessments of knowledge. In the next step of the analysis, we identified, by open-coding, similarities and differences in data and organized these content elements into wider categories, which provided a description to exemplify the findings according to the research questions. We used the concept of recognition as a descriptive tool to unveil what was perceived or viewed as valuable knowledge by the teachers and students within the context of study (Kress and Selander 2012).

Our findings, which emerged from data focusing on depth, have no claims of objectivity or empirical generalization of knowledge associated with nomothetic research (Guba and Lincoln 1994). We are aware that qualitative study research has its limitations, both with respect to the challenge of capturing complex semiotic work and to the lack of external validity and generalized conclusions. Qualitative research such as ours is underpinned by a set of epistemological assumptions quite different from those of quantitative research. Instead, we will argue that our research takes the form of an idiographic inquiry that offers descriptive exemplary knowledge (Thomas 2011), a specific particularity given in context and understood in that context. Furthermore, the study was conducted in line with the ethical principles of the Swedish Research Council (2011). In the beginning of the study, all participants (teachers and students) were informed orally about the aims of this study, the purpose of data collection, and how data analysis will be conducted. The students also received a written information about the study and they all signed a written permission to participate in the study. The students were all eighteen years old and they understood that they could interrupt their participation at any time. During the whole time, we developed a mutually trustful relationship with the students and the teachers.

Findings

Students’ design of multimodal texts

In each project assignment and during the whole working process, the student's design of texts could be categorized as multimodal. The students’ choices of digital tools and modes for representations gave implications of what kinds of knowledge the students could develop.

Log books

In the log books (individual and group log books), the students used different digital tools: Facebook, Google Sites, Microsoft Word and the schools’ Learning Management System (LMS). Each of these tools made use of various modes of communication such as written text, images, sound and video. Despite the availability of these tools, the students log books predominantly contained written text. The written texts were arranged chronologically, similar to the method of diary writing, with a clearly discernible reading path from left to right and from top to bottom. This type of text began with a date and then a description of events and actions related to the topic. Further, the log books were marked by a personal narrative; they were characterized as a sort of blog-writing which gave less attention to applying knowledge of language structure and language conventions, e.g., spelling, grammar and punctuation. The written texts also had a clear mixture of spoken and written language and generally contained many spelling mistakes, as well as grammatical errors and uncompleted sentences. The personal experiences were highlighted and placed in the foreground of their texts. The narrative account was used as a resource for presenting what had happened in the different working phases of their project assignment. Dilemmas and difficult experiences were problematized in some of the student's written texts. At the same time, the written texts foregrounded experiences in a positive sense, for the purpose of showing the teachers that their work processes were successful. There were some differences between the log books generated with Google Sites and Microsoft Word. The texts created in Google sites were longer and closer to the spoken language compared to log books written in Microsoft Word. In all the log books, both narrative and conceptual representations were displayed. Words such as clients, marketing and concept were present in the project assignment on entrepreneurship, while words such as alkali and acid were used in Chemistry. However, the log books mostly involved personal experiences about methodological issues, choices regarding materials and work processes. The written text afforded descriptions of what had been done during the project and how the relations with the other group members had developed, with accounts of temporal and causal links displayed. Even detailed descriptions of places and surroundings where the students met each other in their working process were described in their log texts. As stated above, the students did not use any images, photographs, graphics, or links to videos in their log texts. It was the written text that dominated, although all available digital tools more or less offered others modes of communication. However, the log books gave accounts of how the students had worked with other media and modes within their project assignments. For example, the project assignment about “Life coaching” had taken photos of various persons that they had interviewed. All log books had a clear tone in which the teacher was addressed as the reader. The students handled this differently. Despite the potentials they afforded, Google Sites and the LMS were utilized to a relatively minimal degree by teachers for providing, for example, feedback. There were a few written comments from the teachers on Google Sites.

Written reports

The written reports were written in either Word or Google Docs, and the students’ choices of digital tools shaped the design of the final compositions in the reports. All reports, both in social and natural science, had the overall disposition of an academic text (notwithstanding colors, images, and figures). All written reports were written in line with a preconceived template which offered specific structure that resembled an outline in the scientific genre. The texts showed a variation of compositions, for example in the structure of headings, in the amount of words and in the use of narrative and conceptual languages. The texts switched between different writing genres, some text types related more or less to the academic genre and some to an informal “log writing” genre.

The written reports in social sciences were mainly chronological narratives with less reliance on academic headings such as purpose, methods, materials, and results of discussion. The texts consisted of repeated headers with no clarity of content differences, which affected the coherence between the different parts of the texts. The language styles were also informal and consisted of descriptions of and personal reflections on individual experiences during the project assignment. The texts were also to a limited extent related to theory and many key concepts were omitted without any explanation on the basis of literature in the field. The written reports showed primarily narrative representations and the intended reader was the grading teacher. These student–teacher relations were shown in the texts by different colored markings, personal pronouns, and by description of the student's own efforts in the project assignment. On the other hand, the reports in natural science had texts that were more formal in style and academic in structure compared to the reports in social sciences. The composition of these texts was also more explicit and accurate, in line with the template of the scientific genre, both in relation to headings and to content structure. These texts had clearer objectives, and the presentation of results and conclusion had a mix of written text, tables, graphs, and pictures. However, despite the academic heading structure, the reports had few text types that contained analytical and concluding elements. There were also limited links or references to relevant literature.

Oral presentations

At the end of the course, the student's presented their projects assignments orally in different ways. The teachers in natural science had arranged a seminary that recalled an academic opposition—respondent seminary, while the teachers in social science arranged a different setting. The students in natural science presented their work for 10 min and then another group posed critical questions or comments to the students. In social science, the teachers in the school had arranged a mini-conference at which other, younger students and their teachers had the opportunity to take part in the students’ presentations of their projects assignments. The students had the opportunity to arrange their presentations in different ways. For example, the Life coaching group designed their presentation as an activity in which they first presented their project assignments, and then the audience was offered the chance to participate in a short meditation activity together with the students. The presentations varied in terms of how the student's chose to present their work and which modes and media they wanted to use. Most of the students chose to use a presentation tool such as PowerPoint or KeyNote. As mentioned above, for both the log books and the written reports, the students used predominantly written text. In the oral presentations, the students used various modes of communication in their project presentations, for example both spoken and written language, images, graphs, music, illustrations as well as animations. They designed and combined these modes in different ways to convey meaning in the different settings. In the six project assignment presentations, we identified three different designs for their oral presentations. These were: (1) as a part of regular school work where the students chose to design and sequence their presentation after their written report, (2) as a selling pitch where the students chose to design their presentation with coherence and conscious choice of modes and media, and (3) a combination of the two designs. Below, we will describe an example of the second category. For one of the projects, called Social media and carbon dioxide emissions, a student had constructed a website where the user could calculate what their carbon emissions were when they spent x amount of time communicating through social media. During the oral presentation, the student chose to argue and promote the use of the website and also to convince the audience to think about carbon dioxide emissions when they used social media. The student transformed and contextualized their project assignment about carbon dioxide and social media to render it more in keeping with a “sales pitch” than a presentation of schoolwork, where the audience were potential users of the site rather than merely other participants in schooling. The student also used images, written text coherent with the support or enlightenment of information that was provided. For example, instead of using a strict and correct mathematical formula, the student chose to show images of the numbers and put these together as an animation so that the audience could follow the calculation in a way different to how it was presented it in the written report.

Perceptions of the multimodal texts and recognitions of knowledge

The teachers’ views

The result of the analysis of the teachers’ experiences provides an insight into different ways of perceiving the processes of making multimodal texts and what knowledge is recognized as valuable in the three different literacy practices—log book, written report, and oral presentation.

Face-to-face dialog versus log books

The statements in this category reflects what the natural and social science teachers perceived as the effects of the students’ log writing. The natural science teachers stated that making constantly recurring meetings, for the purpose of reconciliation, was essential for the student's working process. Face-to-face dialog was perceived as more important than giving feedback on the students’ log books. The teachers graded only the student's writing without giving any content or process-related feedback. Instead, they saw the personal meetings in laboratory classes as opportunities to deal with the students’ progress as well as for the students to receive individual feedback on problems and questions. This opinion was more elaborated by the natural science teachers. They argued that it was easier to talk, face-to face, about substantive issues related to both the working process and the writing process.

Excerpt 1: […] they usually realize pretty quickly that it’s quite nice to have a simple question… when they do not have it, it’s not hard to convince them that it is better to have a simple question… just a simple question… but doing it properly… and doing it according to the rules […] (Teacher N1)

The social science teacher was, on the other hand, more optimistic towards communication with the students within social media. The teacher recognized the students’ use of LMS and Facebook as central resources for making the students’ learning processes visible. The teacher meant that the conditions of writing and the students’ learning process varied depending on their choices of media and modes. Some students used log writing as a tool for displaying their own experience and learning process. For the teacher, the written text was not considered as the only way of displaying knowledge for the student. Despite the teacher’s encouraging approach, the students did not use any audio or visual recordings, such as vlogs (video logs) which were an opportunity for the students to choose something other than to write text.

Making the expectations transparent of what is acknowledged in the reports

The natural science teachers were mainly focused on the written report as a product. They saw the students’ text-making processes as formal and purpose-driven writing processes. They recognized texts in the linguistic mode, which were part of the subject domain in natural science. The teachers were concerned about the reports’ accuracy related to content and style and how different text types in the reports were formally and linguistically appropriate to an academic purpose. The teachers’ conceptions of what constituted a written report within the natural sciences reflected their teaching and supervision approaches.

Excerpt 2: […] then I explain exactly how I want them to do it […] I really like them to use EndNote or something but they cannot do that, it had been fashionable since then so I’m going through exactly how I wanted the summary to look, the introduction, the materials and methods and results section and the discussion[…] (Teacher N1)

Examples of ideas and old reports written by former students were given to the students during supervision. The teachers were also concerned that all students’ ideas needed to be approved by teachers before starting the project assignments. Teacher N1, chemistry teacher, was even more concerned with establishing close contact with the students in which students needed to communicate their tentative ideas and come to weekly scheduled laboratory lessons. According to the teachers’ experience, the student's work would never be successful if the teacher allowed them to develop ideas all by themselves without any strict guidance. The requirement of learning academic writing was something that was recognized as an obvious literacy skill for the students to learn and practice. One of the teachers highlighted the academic writing practice as a central scholarly skill in natural science which had to involve a process of self-regulation and practical planning, drafting and revising text, and continued communication with the supervising teacher. Teacher N2 explained that these academic skills are often shaped over a long period of time in education as a whole. The students needed to practice these skills and competences by regular paper writing after work in the laboratory. Academic skills, such as conducting small but researchable investigations were also seen as something that was shaped by the student's interests and by the talk and feedback from the teacher in the laboratory classes.

Excerpt 3: […] they come to the class and want to do something… anything… with cancer and there’s not a chance and that’s not possible, to do it… and sometimes they have also quite extravagant ideas of how they want… for example, we have quite a few groups that make perfumes and when … they love to extract the essential oils from things and they do not have any realistic ideas … that they needed 300 kilograms of rose petals for getting a few drops of rose oil like that … but they realized damn fast … they came to know the limitations themselves very quickly[…] (Teacher N2)

The views of writing the report were also concerned with the process of scientific work within the project. One dominant recognition or criterion for what counts as knowledge in the project assignment was finding a researchable idea. This idea was later recognized in grading by the natural science teachers, if the writing and the usage of methodology was appropriate to the purpose they served in the report. In contrast to this more explicit recognition of getting the students into an academic socialization, the social science teacher, on the other hand, viewed the writing of the report as more related to the students’ own interests and creativity. The teacher believed that the students had their own power and inspiration to identify an idea and to come up with methods and materials for the purpose of the project assignments.

Excerpt 4: […] Yes, one in each group came to me…each one came up with the idea that if I got it right, and so it spread, and it was awesome for the boys, I think three of them are in one class and one in another class […] and they hang out after school… a little bit but… we succeeded really in interesting all four, they are interested in different things in the project (Teacher S1)

It was neither formal nor informal expectations which pointed out for the students what was required in the report writing. The teacher determined that the students’ writing process improved by implicit teaching from the teacher and without any feedback or corrections of style, spelling, and grammar. The social science teacher’s view of writing and teaching the students to write was more related to conversations about topics of writing and processes of working in the project. The teacher saw the students’ writing, both in the reports and in the log writing, as something that was related to creativity and self-expression. The purpose of writing was nearly the same. The social science teacher believed that the students wrote more and worked better without any strict guidance. There were no issues or concerns about researchable ideas.

Independence: the core competence

Statements in this category reflect what the three teachers perceived as the core competence the students needed to develop within the project assignment. This competence was viewed as a phenomenon that they called “independence”, and was seen as something that was represented in the students’ self-regulating actions. For example, as accounts of how they become owners of their own learning, and such representations of valuable knowledge were displayed both in the log texts and in the personal sessions together with teachers.

Excerpt 5: […] I said come to me, as if you were responsible for a project at work and that you needed to report to the manager…how it goes… and now I have come this far and have had these difficulties. Keep me informed, please contact me, let me be involved in the process, ask me for advice if you need to. And the responsibility lies on you […] (Teacher N1)

At the same time, the teachers expressed an understanding of independence as something that required knowledge and skills with regard to how to identify interesting questions and the ability to carry out scientific work by themselves. They saw such knowledge and skills as prerequisites for the representation of independence. The students who had not been able to make such knowledge or skills of independence visible received lower scores from the teachers.

High grades require equally subject-related content skills and process skills

Statements in this category reflect what was recognized and graded as valuable knowledge by the teachers. None of the teachers viewed the quality of the written reports alone as the main representation of recognition and criterion for grading. They all recognized both the subject content skills and the process skills learned in the project assignment. The natural science teachers had also developed a quantitative dividing system of assessing and grading the process and the products: log books, written reports and oral presentations.

Excerpt 6: […] I weight them, so I put five points on each thing, then weight them, but… it’s probably the working process I emphasize most…. it is more how are they working?, how do they interact with me, how do they think? [..] (Teacher N1)

However, none of the teachers gave high importance to the oral presentations, believing that the oral presentations were difficult to assess and grade. Even the natural science teacher who organized the oral presentation as an academic defense session, did not include the students’ oral representations in the assessment practice. Nevertheless, the students needed both to defend their own project assignments and give feedback to other students. The students’ multimodal text productions, their speech in combination with other modes, for example font, color, graphics or images, were not emphasized equally by the teachers. The teachers only noticed the single mode of speech, which they also believed was not recognized as assessable. They lacked grading criteria on speech performance. They were afraid of assessing the student's social skills.

Excerpt 7: […] I cannot assess that …I can't assess their nervousness really and I do not… because I can't, I have even gone so far, that if there is someone who absolutely does not want to do this …feels so bad they lie there and vomit for three weeks before, it must be a hell for them… I can't grade this, I think that it’s not, it’s not my job to do that […] (Teacher N2)

The students’ representations of knowledge in modes other than speech, such as use of written text or images in Power Point slides, were disregarded and neglected by the teachers. The teachers mainly assessed the students based on two foci: the students’ ability to make the working process skills visible in log books and in supervision and their ability to present the whole project assignment, from the concept to the final product in written reports. These representations of knowledge, which were displayed both in a linguistic and visual design in log books and written reports, as well as in an auditory design in oral communication within supervision, were equally recognized by the teachers.

The students’ views

The result of the analysis of the student's experiences provides insights into different ways of perceiving the processes of making multimodal texts and what knowledge is recognized as valuable in the texts.

Frames: course plan and course criteria

The students highlighted that they were well aware of the assessment criteria. They stated that they had learned about the learning objectives, the course plan, and the criteria at the beginning of the course and also that the course criteria had been handed to them in the beginning of the course. However, the students conducted themselves differently with respect to the frames of what was given to them through the course plan and criteria. One of the students did not need to read the criteria “word by word” because they already knew what specific competences and skills were required for achieving higher grades. They said:

Excerpt 8: […] You learn a certain way to think to get the higher grades. (Student 3, biology)

This student was aware of the learning objectives, and of what kind of knowledge and texts were required as valuable during the learning process within the project assignment. The student stated that it was a competence he/she was schooled in. Using the syllabus and the criteria was a way of thinking of what was recognized as valuable and important knowledge in the school setting. The student saw this as a strategy that all students usually used, as a thinking device, throughout the course of their education. However, another way of thinking about the learning objectives and the criteria was putting them aside and instead focusing on a personal interest. One student stated:

Excerpt 9: […] Instead of following the course plan I rather do something that I'm passionate about so I can deliver a good final product (Student 11, entre)

This way of perceiving the learning objectives is rather different from Student 3. Student 11 emphasized that he/she was learning for life, not for achieving high grades in school. The students were aware that the requirements in the school setting and the challenges in “real life” were two, more or less, separate and different goals. For example, one of the students said that he/she would not have published the school report as it was on a public website. If they had published the report outside the school, they would have written it differently. Yet another student framed the tasks in the project assignment as far more than “just a school assignment.”

Excerpt 10: […] The only thing I know is that we have undertaken a journey that some of us start way later in life and that is good enough for me and cannot be replaced by any grade. (Student 10, entre)

In the interview, we asked the students to grade their work in relation to criteria in the course plan. One of the students said: I will get the highest grade because of my analytic ability in the final written report (Student 3, biology), which shows that the student knew what knowledge was recognized as valuable by the teachers. All the students gave an accurate self-assessment about what grade they would receive on the project assignment at the end of the course.

Supervision versus independence

There seem to be a common understanding among the students that if they received a lot of supervision, the grades would decrease. They were referring to the course criteria concerning independence, and they connected this to how much supervision they had in a quantitative way. However, how much supervision they could receive while still being recognized as independent was unclear for them. However, they were confident that the teachers recognized qualitative aspects in the communication more than how many times they utilized the supervision as a learning resource. One group had a special relation towards their supervisor and therefore understood supervision differently. They did not reflect upon independence in the same way as the other students. They saw the supervision as a steering function so that they could keep their work as a “school-assignment” on track.

Excerpt 11: […] We had our idea and we just did it and then she (the supervisor) was there and made sure we followed the course plan. (Student 12, entre)

Multimodal texts: explicit on the working process: implicit on the written report

The students stated that the teachers emphasized and assessed foremost the working process. Nevertheless, it was unclear what the students meant when they referred to the “working process.” Two of the students said that the process was connected to problem solving, and that they could think for themselves (Student 2, biology). The students also mentioned supervision as an important part of the process where they meet physically and had conversations about the multimodal texts with the teachers. However, the frequency of supervision should be kept to a minimum in regard to the students’ independence. Some of the students mentioned the writing of the log books as a task that more or less had to be done in order to get their grades—not as a tool for learning and reflecting about their learning process, but simply something they needed to do. One student mentioned that she/he would not have written the log books if it had not been a mandatory requirement of the course. The student did not see the point of writing logs texts. However, some of the students saw instead the log texts as an important representation for themselves in their working process. Two of the students said that their process and reflections about their work became visible through their writing. We had to write our plan in the log-text and I think it was rather good (student, 2, biology). The written reports were the representations that the students recognized and saw as most important for the final grading of the project assignment. The oral presentation and the multimodal texts within it were the least recognized representations for grading by the students. However, all students believed the presentation were fun to do and they appreciated other students’ presentations.

Discussion

If we return to the beginning of this article, to the aim and the research questions, we have illuminated how multimodal texts were designed in three literacy practices and which representations were recognized as valuable knowledge by teachers and students. The findings show that the students were designing their interest and understanding of the task in relation to different digital tools (Facebook, Google Sites, Microsoft Word, PowerPoint, and the schools’ Learning Management System) and that the students had chosen foremost the linguistic mode to represent their knowledge in the three literacy practices (Tables 3, 4, 5). In the log books, the student's personal experiences were highlighted and placed in the foreground of the written texts. The narrative accounts were used for presenting what had happened in the different working phases of the project assignments. Dilemmas and difficult experiences were problematized but even experiences in positive senses were described. The written reports were overall formal in style and academic in structure. The reports in natural science were more explicit to academic conventions compared to the reports in social science. These texts had clearer objectives and the presentation of results and conclusion had a mix of written text, tables, graphs, and pictures. The linguistic mode was especially dominant in the student's logs books and written reports. However, the visual and auditory mode became more vividly orchestrated by media in their oral presentations. In the log books, a personal and process-related narrative was more frequently used, and in the written reports, an academic conceptual usage was more dominant. It was only the oral presentations that had a varying balance between different modalities. In the oral presentations, the students combined and orchestrated different modes. They used speech, music, text, images, and colors in an intertwined mix. Some of the students in the project assignments, Life coaching and social media and carbon dioxide emissions, designed the multimodal texts related to a creative self-expression (cf. Ivanič, 2004). The student's interests and understandings of the task were shaped in a complex uses of modes, and they developed personal voices and something interesting by their own choices. This result confirms previous research (Kjällander 2011; Öman 2015; Sofkova Hashemi 2017) that multimodal texts production may emerge implicitly through meaningful participation and not primarily through a teacher-led instruction. The multimodal texts production, within the oral presentations, can be seen as a purpose-driven activity, where the complex usage of modes and media is a part of an already acknowledged and appreciated literacy practice.

Table 3 Digital tools and design (linguistic, visual, auditory) elements in relation to the log books
Full size table
Table 4 Digital tools and design (linguistic, visual, auditory) elements in relation to the written reports
Full size table
Table 5 Digital tools and design (linguistic, visual, auditory) elements in relation to the oral presentations
Full size table

During the supervision and within the whole design for learning process, the students were engaged with several media and modes when they were learning to write. They were constantly forming and changing the meanings of subject content from one mode into another, e.g., they talked about the project assignments and they wrote about them in their log books and the written reports (Kress and Selander 2012, Jewitt 2009). The meaning was sometimes partly altered and shaped differently in one mode compared to another. A specific piece of content in the written text was given a somewhat different meaning by addition of intonation and narration in the oral presentation (Table 5). We also find indications that the texts by the students in natural science (from school B) were more formal and academic in register compared to the texts by students in social sciences (Tables 3, 4, 5). The composition of the natural science texts, within all three literacy practices, were also more pure in relation to writing within the scientific genre. The student's use of the linguistic mode was formal and had a clear objective of inquiry. The texts also combined other modes such as tables, graphs, and pictures with the linguistic mode. On the other hand, the texts produced by the students in social science (from school A) were more vivid in social expressions. In these texts, in the linguistic mode, they gave expressions of subjectivity and relational positioning using pronouns and color markings for the purpose of displaying social relations and individual responsibilities. The students were positioning themselves and each other more freely compared to texts produced by the students in natural science. A final remark on the students’ text productions is that the linguistic design dominated over the other design modes.

Furthermore, we see the student's selection of digital tools and modes as an effect of social and cultural traditions in writing and assessment of multimodal texts (Kress 2010). None of the students, in either school A or school B, used any digital tools which enabled audio or visual recordings. The linguistic design seemed to be the most common way for the students to choose when they were engaged with modes for the purpose to represent knowledge in the project assignments. The linguistic mode seemed also to be the most appreciated one, especially by the teachers. Although teachers said that they recognized the student's working process and their products of writing equally, it was still the written text that was seen as the central and the most valuable mode of representation of knowledge when it came to grading. None of the teachers knew how to assess the oral presentations, especially not the students’ speech or how they can value all texts in relation to each other. The teacher's and student's perceptions revealed beliefs that traditions are by far the most controlling and sometimes constraining factor in developing new forms of writing. The study showed that the students design of multimodal texts (both in social science and in natural science) and the teacher's and students’ views were affected by ideas which were formed by local scopes and educational traditions (Kress and Selander 2012; Ivanič 2004). Such ideas shaped the conditions and had implications for the students’ designing processes and how teachers and the students viewed and recognized the writing of multimodal texts. All teachers and students viewed writing in log books as something valuable in its own right. They viewed and recognized the writing of multimodal texts as part of a creative process which was purpose-driven in a relation to the requirements in the curriculum and to the specific context at the schools. The natural science teachers showed stronger academic recognition of the written reports compared to the social science teacher. They considered that the written reports ought to be shaped by subject and academic traditions. Tables 6 show that the teachers in natural science emphasized the academic genre and structure to a higher degree than the social science teacher did. These views were evident both in the student's representations (written reports and oral presentations) and in the interviews with the teachers (see for example Excerpt 2). In social science, the teacher emphasized the creativity and writing as a local practice, and in natural science the teachers emphasized writing as a process to develop academic skills. The digital tools used by students were also more or less selected to suit the academic purpose of a linguistic and an auditory design. We argue that the selections of media and modes were foregrounded without any critical reflection of the potentials of used media and modes. None of the students made any audio or visual recordings. We will argue that all these design processes were effected by different traditions, which are formed by different cultures of recognitions and assessment practices of writing and multimodal texts (Kress 2010; Ivanič 2004) Table 7.

Table 6 Recognition of knowledge in log books, written reports and oral presentations
Full size table
Table 7 Recognition of knowledge in log books, written reports, and oral presentations
Full size table

Concluding remarks

This article has presented a study in which student's multimodal texts production in three different literacy practices has been explored. The article has also presented how students and their teachers perceived and recognized the multimodal texts produced in these literacy practices. The findings presented from this study show that the students mainly used linguistic design for representing knowledge despite the digital opportunities to include for example images and sounds in their representations. The linguistic design was more recognized by the teachers and students and valued in relation to the practice of assessment and grading. As digital technology becomes more common in today's classrooms, the multimodal texts are also becoming more visible and need to be recognized in the students’ writing. Digital technologies enables inclusion of multimodal text, such as making written text, images, vlogs, podcasts be organized equally, which challenges the understanding of students’ writing and even the recognition of knowledge and competence in many ways. This article's results reveal the urgent need to challenge students in the design process of making multimodal texts, and even towards developing the teacher's teaching and their assessment practices such that text production encompasses a pedagogy of multiliteracies. By providing that perspective in teaching, new and challenging values and knowledge can therefore be developed and recognized by teachers and students in assessment. The findings open up several other research questions, relating to exploring assessment practices and cultures of recognition in schools (Kress & Selander 2012). We see a need to further investigate and develop assessment tools and criteria in order to recognize students multimodal texts in formal education. Finally, we would like to emphasize that the framework of design-oriented multimodal perspective (Selander and Kress 2010; Kress 2010) gave us means of representing designs and pinpointing patterns of belief about writing and recognition of multimodal texts. This perspective might also give teachers in schools analytic tools for the purpose of grasping a broader understanding of students’ writing and recognition and assessment of multimodal texts. We argue that this perspective provides the meta-language required for new forms of communication competences. It can be used critically for formative and summative assessment of multimodal texts. We hope that this study can be used as an example of how this framework (cf. Selander and Kress 2010; New London group 2000; Hung et al., 2013; Sofkova Hashemi 2017; Huang and Archer 2017) can be used as a metalanguage for detecting and viewing design approaches to multimodal writing and assessment of multimodal texts in schools.