This chapter presents the evaluation findings from over 2500 classroom pilots of tools and resources designed to support the development of digital pedagogy. The iTEC approach is an innovative process to support scenario-led learning design. Data collection included surveys, interviews, and classroom observations from teachers, students, policy makers and other stakeholders. This chapter focuses on the impact of iTEC on digital pedagogy; 12 key findings are presented in relation to learning and learners, teaching and teachers, and the potential for system-wide adoption of the iTEC approach. These findings suggest that through participating in classroom pilots: students developed twenty-first century skills; students’ roles changed; there was a positive impact on students’ motivation; and students’ attainment was positively affected. Furthermore, through participating in the project teachers enhanced their digital pedagogy; became more enthusiastic about their pedagogical practices; increased their use of technology; and collaborated more. With refinement, the scenario-led design process could support mainstreaming of innovation. The library of scenarios, Learning Stories and Learning Activities was perceived to be a valuable output. Towards the end of the project there were growing signs of awareness and uptake, particularly in countries where the approach aligned closely with current policy direction. The chapter concludes with recommendations for policy-making, the management of teaching and learning, technology provision and research.
Introduction
This chapter presents the evaluation of the large-scale piloting that took place from September 2011 to June 2014. Through iTEC, educational tools and resources were piloted with around 50,000 students in over 2500 classrooms (exceeding the original target of 1000) across 20 European countries. This chapter synthesises the evidence of the impact of iTEC on learners and teachers, and the potential for system change, looking at: iTEC processes, tools and resources; classroom perspectives; and national perspectives.
European educational policy (such as Europe 2020) includes as one of its targets increasing employability and life-long learning through developing students’ digital competency. There is also a need to develop students’ twenty-first century skills which are increasingly important in the workplace (Dede 2010; Redecker et al. 2011). The majority of European teachers are using ICT primarily for lesson preparation; use in lessons with students is still limited despite infrastructure having improved substantially (EC 2013). There is, thus, a growing need for teachers to be supported in the development of digital pedagogy through learning design, an approach which is growing in importance but not yet widely adopted (Emin-Martínez et al. 2014). It is widely asserted that, in order to remain competitive in global markets, education and training needs to be transformed; one way to address this is to mainstream the use of technology for learning and teaching through national policies (EC 2012; Brečko et al. 2014). Given that uptake of digital pedagogy is still low, it is essential to explore mechanisms that can support system-wide change (Brečko et al. 2014). The iTEC project set out to address this through the development of processes to support such needs. The resulting iTEC approach involves the development of Future Classroom Scenarios, and the Learning Activities that are derived from them, to inspire teachers to develop digital pedagogy.
Anzeige
The iTEC evaluation addressed three key questions:
How did the iTEC approach impact on learners and learning?
How did the iTEC approach impact on teachers and teaching?
What is the potential of the iTEC approach for system-wide adoption in schools?
The mechanisms for scaling-up pedagogical change through technology integration included: a learning design process (see Chaps. 3 and 4); professional development for teachers; and support systems such as online communities. The aim was to develop pedagogy enabled through, rather than driven by, technology innovation. This has been found to be critical to effective adoption of technology-enabled learning (Ertmer and Ottenbreit-Leftwich 2013; Kampylis et al. 2013). The iTEC approach does not focus on specific technologies, nor even digital pedagogies. It is designed to account for a constantly changing technology landscape, and enable learning design to respond the current context (at many levels in including policy, national and school).
Scenarios (see Chap. 2) were developed through bringing together a wide range of stakeholders (including teachers and students) to identify current educational trends, together with collaborative workshops tasked with developing responses to such trends. Learning Activities (see Chap. 3) were developed, in a participatory process involving teachers, by identifying design challenges, then addressing them through selecting resources and developing prototype tools. The iTEC project also developed a number of prototype technology tools to support the learning design process and classroom activities (see Chaps. 4–8).
Piloting was supported at national level by pedagogical and technological coordinators who recruited teachers, provided training and facilitated online and face-to-face communities and workshops, and undertook aspects of data collection for the evaluation. A five-day face-to-face professional development course, comprised of a suite of iTEC modules and training materials, was created. The course can be localised and adapted for use at national and regional level. These resources were also adapted for a short course and for a Massively Online Open Course (MOOC).
Anzeige
In the first four cycles, teachers were presented with a package of Learning Activities, exemplified through 2–3 Learning Stories (see Chap. 2). These were created centrally (involving a wide range of stakeholders) and subsequently localized by national coordinators. Localization in some cases involved a selection process at national level which meant that teachers had little, or no, choice (i.e. teachers were presented with a single Learning Story and accompanying package of Learning Activities). As iTEC technologies became available, teachers were encouraged to incorporate them into their piloting activities. Across the four cycles Learning Activities included twenty-first century skills (independent learning, critical thinking and problem solving, communication and collaboration, creativity, ICT) integrated with project-based approaches, teamwork, reflection, peer assessment, outdoor learning, involving outside experts, and students as designers and producers.
iTEC in practice: Implementing the Redesigning School Learning Story, cycle 3, UK
This Learning Story required students to think about spatial design and the different motivations of people who use a particular learning space. The aim was to design a new space for future use based on identified current challenges in relation to school-based activities. Implemented in a UK secondary school as part of a Product Design course, it took 10 lessons over a period of 5 weeks. Students were divided into groups of three using TeamUp (an iTEC prototype technology). Before they started, students agreed the class ground rules and their team roles. The teacher created an Edmodo group (a social learning network designed specifically for education) to allow students to share their work, receive group messages and access resources in the ‘library’. Students were presented with a design brief that the teacher had created, and were allowed to use their own mobile devices to record the issues they found around the school. They then used their own tablets to record photos, videos, make notes and record their thoughts and reflections throughout the project. Students without tablets were loaned portable video cameras. Students created a prototype and then discussed their design with future users. Based on the feedback, students then created their final design prototype, which they presented to the class. Perceived innovation included students working as producers, collaboration, easy collection of multimedia data and students developing a better understanding of the design process.
In the final cycle of the project, coordinators in each participating country facilitated the learning design process (rather than this being facilitated centrally), running workshops for scenario and Learning Activity development that involved a wide range of stakeholders including students and head teachers (in excess of 700 across both processes, the majority of whom were teachers). In this cycle, coordinators were asked to incorporate an iTEC tool for learning design (see Chap. 6) into the Learning Activity development process and to encourage teachers to use other iTEC tools (see Chaps. 5–9) either in their classroom activities or through workshops. iTEC technologies were developed to support the design process, to curate digital resources and to connect teachers.
The main outputs of the project were: a scalable scenario-led design process for developing digital pedagogy; the Future Classroom Toolkit and accompanying training provision; and an extensive library of Future Classroom Scenarios, Learning Activities and Learning Stories.
Background
Reflecting on the landscape from the conception of the iTEC project to date there have been many changes. In 2010, social media use was comparatively rare; it is now more prevalent (Aceto et al. 2014) although teachers and students still require support to use it safely in schools and to develop their skills to maximise the impact on learning (Wastiau et al. 2010). Although research evidence is limited, there are indications that social media, combined with student-centred approaches to learning, can positively impact on student achievement (Hew and Cheung 2013). There has been a huge increase in the use of tablets and smartphones since 2010, both in day-to-day life, the workplace and education (Purcell et al. 2013; EC 2013). The use of mobile devices is perceived to be important for innovation in secondary school classrooms (Aceto et al. 2014).
Game-based learning continues to be ‘on the current horizon’ (Groff 2013; Johnson et al. 2014); this remains unchanged. It is interesting to note that whilst its proponents remain optimistic, even evangelistic, uptake remains limited. The evidence on the relationship between games-based learning and impact on ‘academic achievement’ is mixed, but there is consensus that such use can impact positively on ‘problem solving skills, broader knowledge acquisition, motivation and engagement’ (Perrotta et al. 2013:ii). The potential of gaming and gamification warrants further research to understand why it is not being adopted by teachers. Game-making in particular has potential to support the development of computational thinking, another twenty-first century skill that advocates claim is important for life beyond education (The Royal Society 2012; Grover and Pea 2013). Game-making can lead to improved understanding of subject knowledge, creativity, increased engagement and the development of problem solving skills, critical thinking and deep learning strategy use (Vos et al. 2011; Yang and Chang 2013).
However, the typical use of technology to support teaching and learning remains rather unadventurous, confined largely to using office tools and internet searches; digital pedagogy is still undeveloped in the average European classroom despite improved provision of infrastructure and other resources (EC 2013).
Evaluation Questions and Approach
The evaluation was designed to support the development of the iTEC approach and prototype tools, as well as to assess impact on learning and teaching. Therefore, formative, rather than summative, evaluation was necessary, underpinned by qualitative data collection. Learning Activities and Learning Stories were sources of inspiration for teachers to own and adapt, rather than a fixed series of prescribed actions, resulting in wide-ranging interpretations and implementations. Given the diverse nature of the pilots, the project could not set out to provide quantitative measures of impact on student performance.
Regular surveys of teachers and learners yielded perceptions about the impact and future potential of iTEC outputs. Teachers’ opinions about whether or not an idea ‘works’ for them are important (reflecting their experiences, understanding of the complexities of the classroom, and the particularities of their context), as are indications of intended future use (Dillenbourg and Jermann 2010; Voogt et al. 2011). Case studies, including interviews with relevant stakeholders (e.g. teachers, students, head teachers) and observations of lessons, enabled the particularity and complexity involved in the implementation of Learning Stories to be explored (Stake 1995) and provided an opportunity to triangulate teachers’ claims against observed practices. In order to strengthen the evidence further, national case studies involving interviews with policy makers and key stakeholders were conducted. Assertions that are warranted by a wide range of data sources are stronger than those warranted by a single data source, irrespective of the number of ‘instances’ of such data (Erickson 1986). Therefore, collecting data representing a wide variety of stakeholders’ perspectives about their experiences of the iTEC approach increased the robustness of the evaluation approach adopted.
Data were collected (September 2011 to June 2014) as follows:
68 implementation case studies (interviews: teacher, head teacher, 6–8 students, ICT coordinator; lesson observation);
16 national case studies (online interview with two policy makers and the MoE partner lead)
National coordinators arranged for the surveys to be translated into national languages. Surveys were administered centrally using an online survey service. Data collection for classroom pilots and iTEC processes, tools and resources was undertaken by national coordinators. Co-ordinators were provided with written guidance on evaluation procedures for each cycle together with an online training session. Coordinators were also encouraged to seek advice as and when required. National case study interviews were conducted directly by members of the iTEC project team. Whilst the analyses of these interviews are presented as ‘national case studies’, of course they actually only reflect the view of 2–3 stakeholders, albeit directly or indirectly related to national policy making.
The evaluation has thus utilised a variety of data collection approaches and gathered the perspectives of a wide range of stakeholders including teachers, students, national coordinators, policy makers, head teachers, and school ICT coordinators. Moreover, it has taken place over the course of 3 years, embedded within a cyclical design, which enabled the iTEC approach and iTEC prototype technologies to be tested and refined.
A responsive approach to the evaluation was undertaken, refocusing in the later stages to: capture and document the innovative iTEC processes that could support mainstreaming; to shift the focus of evaluation from classroom impact to strategic impact; and to place greater emphasis on the evaluation of iTEC technologies.
How Did the iTEC Approach Impact on Learners and Learning?
The iTEC approach concerns Future Classroom Scenarios and the systematic design of engaging and effective Learning Activities involving innovative digital pedagogies. Here, we report on how iTEC impacted on the learner’s classroom experience. Learners engaged in Learning Activities including group work, reflection, peer feedback, product design and producing digital (and other) artefacts, using digital tools.
Key finding 1: Teachers perceived that the iTEC approach developed students’ twenty-first century skills. Their students had similar views.
Teachers and students agreed that engaging in iTEC Learning Activities developed students’ skills (see Fig. 9.1). 85 % of students (n = 1488, cycle 5) agreed that they became more confident ICT users and 86 % agreed that they could now use a wider range of new technologies.
Fig. 9.1
Perceived positive impact of iTEC on students’ 21st century skills. The percentage of teachers (n = 573–594) and students (n = 1444–1488) in agreement, cycles 4–5
…the fact that classes became more appealing, and that it developed pupils’ critical thinking. They began learning to listen, argue, which was something they were not used to doing; they learnt to address their own views in a relative manner and to accept the ideas of others. Then they began gathering different points of view, reflecting and making decisions. This is very innovative. (Portugal, teacher interview, cycle 5)
×
The positive impact of the iTEC approach on the development of students’ twenty-first century skills replicates findings from similar studies such as the impact of digital storytelling (Niemi et al. 2014) and one-to-one laptop provision combined with a shift to student-centred pedagogies (Lowther et al. 2012). Generic skills become increasingly important as learning becomes more student-centred, social and collaborative (Redecker et al. 2011). Assessment systems should be revised to better account for twenty-first century skills and key competences (Brečko et al. 2014). Introducing effective pedagogical approaches, together with policy reforms, will ensure that students leave education with appropriate skills for the workplace.
Key finding 2: Student roles in the classroom changed.
According to the teachers surveyed, the most common way in which iTEC had made a difference to their pedagogy was that students’ roles changed. More specifically, teachers referred to increased independent learning and student autonomy. This change was also noted as an important pedagogical innovation in the national case studies.
…you give them free rein throughout the project. People work at very different speeds and do very different things. So I have to give up some control here. I must. I have to rely on the students to actually do the job even though I can’t see them all the time. (Norway, teacher interview, cycle 4)
Through the changes in student-teacher roles, learners became ‘teachers’ by means of a variety of activities including as peer assessors, peer tutors, teacher trainers and co-designers of learning.
In a group there are always some students who do not know quite what to do and another student will explain; they seem to learn better [this way] than when I explain even with the same words. (Spain, teacher survey, cycle 5)
The majority of teachers agreed that iTEC Learning Activities enabled students to engage with complex, real-world problems (76 %, n = 595, cycles 2–3) and to have opportunities to learn beyond the boundaries of the classroom (86 %, n = 826, cycles 1–3).
An important feature of the iTEC approach for a number of teachers was that it offered students more authentic learning experiences, which more closely reflected situations they were likely to encounter in the workplace, and in later life more generally. These included, working in teams, working with external partners, and producing work that would be seen, and used, beyond the school. The use of technology to bring the outside world into the classroom was also viewed as beneficial.
The development of student-centred and project-based, hands-on, real-world experiences, together with student collaboration, are becoming increasingly important globally, necessarily influencing student and teacher roles (Redecker et al. 2011; Johnson et al. 2014). Technology can enable teachers to more easily support authentic learning through, for example, facilitating greater access to resources and experts in the field, data recording, recording reflections and sharing ideas (Lombardi 2007; Laurillard 2012). As indicated above, the iTEC approach can help students to adopt new roles, collaborate with peers, and engage in authentic learning experiences, all supported through technology.
Key finding 3: Participation in classroom activities underpinned by the iTEC approach impacted positively on student motivation.
The positive impact on student motivation was the strongest theme emerging from the data as evidenced by survey and qualitative data.
Teachers and students agreed that engaging in iTEC Learning Activities positively impacted on students’ engagement and motivation (see Fig. 9.2).
Fig. 9.2
Perceived positive impact on students’ engagement and motivation. The percentage of teachers (n = 826–1399) and students (n = 1444) in agreement, cycles 1–5
Overall we liked the lesson very much. The level of engagement and motivation was quite different. Peers that normally do not participate very much got involved and that was very new. (Austria, student interview, cycle 3)
Pupils love activities connected with using modern tools and creating a PC game was a thing that was really motivating for them. So from my point of view, the greatest thing was the interest. (Czech Republic, teacher survey, cycle 4)
×
This finding accords with other recent research on teacher perceptions of the impact of technology use in the classroom on student motivation and engagement (eg Pegrum et al. 2013; Perrotta 2013). Indeed, research on the impact of ICT teaching and learning frequently refers to increased motivation and engagement (Condie and Munro 2007).
Key finding 4: The iTEC approach improved students’ levels of attainment, as perceived by both teachers and students.
67 % of teachers (n = 1399, cycles 1–5) agreed that the iTEC process improved their students’ attainment in subjects, as evidenced by their assessment data.
We had possibilities to improve our practical skills. We liked working together, collaborating, creating web-pages, photos, film. We have got a lot of positive assessment, high scores—it’s especially inspired us. (Lithuania, student interview, cycle 1)
My French is not very good, I cannot read and speak it that well. But in this course it went better because I was being filmed. I wanted to do it really well. (Belgium, student interview, cycle 3)
iTEC has led to significant improvements [in students’ learning outcomes through creating a deeper] understanding of a topic located in the curriculum and [relating it to] daily life with the use of technology. (Turkey, teacher survey, cycle 4)
It should be noted that the data gathered in relation to impact on student attainment focused on perceptions (although teachers were explicitly asked to respond on the basis of their assessment data) and has not taken direct account of formal assessment data.
There is compelling evidence that the use of ICT in the classroom can have a positive impact on student attainment (eg Tamim et al. 2011; Cheung and Slavin 2013) although of course many factors can influence this such as subject area, type of technology and teacher experience. In common with general evidence, although based on perceptions, both teachers and students agreed that student achievement was positively affected by technology use in iTEC.
How Did iTEC Impact on Teachers and Teaching?
Teachers participating in iTEC were involved in learning design processes and implementing Learning Stories and Learning Activities with cohorts of students. Thirty-six detailed scenarios were developed in cycles 1–4 by a small number of teachers who were managed centrally. A further 22 scenarios were created in cycle 5 by larger numbers of teachers using a standalone toolkit and managed at national level. Another 14 scenarios were created through a centrally-run training course and by an expert group. In cycle 5, a wide range of different Learning Stories and Learning Activities were also created through workshops held nationally.
Key finding 5: Teachers perceived that the iTEC approach enhanced their pedagogy and digital competence.
Teaching creatively involves experimentation and innovation, and making learning exciting through imaginative (and sometimes unexpected) approaches (Jeffrey and Craft 2004; Education Scotland 2013). ‘Creative classrooms’ include ‘innovative practices such as collaboration, personalisation, active learning and entrepreneurship’ supported through digital pedagogies (Bocconi et al. 2012:4). Thus teaching creatively demands change, and the incorporation of digital tools (requiring the development of digital competences) to support new pedagogical practices is one way of achieving this.
Facilitating iTEC Learning Activities enabled teachers to develop their pedagogy (see Fig. 9.3).
Fig. 9.3
Perceived positive impact on pedagogy. The percentage of teachers (n = 826) in agreement, cycles 1–3
×
iTEC was perceived to lead to increased creative teaching.
Now I’m way more convinced of the need to push the school practice in this direction, because this enriches the students, offers new learning possibilities, and makes my teaching more interesting. (Italy, teacher interview, cycle 2)
New forms of assessment were implemented by many teachers including peer feedback, reflection, self-assessment, online assessment and the assessment of digital artefacts. For example, reflection through blogs enabled teachers to monitor progress, developed students’ metacognition and self-evaluation, and supported peer learning.
The significant progress was peer assessment—helped us greatly to see our work in the eyes of colleagues and examine our progress. (Israel, student interview, cycle 2)
Implementing Learning Stories in the classroom encouraged teachers to innovate and experiment. Furthermore, 88 % of students (n = 1488) agreed that their teacher was using different methods to help them learn.
Teachers (cycles 4–5, n = 583) were asked to rate how different their pedagogy was when implementing the Learning Story, in comparison to what they were doing before, on a scale from 1 (not at all) to 10 (radically different). 28 % of teachers stated that their pedagogy had changed substantially (a score of 8–10). One in four teachers (25 %) perceived that their pedagogy was not markedly different to their previous teaching methods (a score of 1–4). This is unsurprising given that there was a bias towards teachers who perceived that ICT competency level was high; teachers with greater confidence are more likely to volunteer to participate in projects such as iTEC.
There was evidence of the positive impact of iTEC on teachers’ digital competence throughout the project. Qualitative data echoed that of survey data in relation to the development of ICT skills, including digital pedagogy.
The project invites me to use more new technologies and suddenly you feel more comfortable and they can be used more easily. This is what I found. (France, teacher interview, cycle 4)
Learner-centred pedagogies are essential given the growing importance of the knowledge society (Voogt et al. 2013). Current technologies readily support learner-centred activities such as collaboration and communication, and can thus easily support such pedagogical shifts (Beetham 2013). However, it should be noted that repeated attempts to change classroom pedagogy through educational reforms have not been successful (Cuban 2013). Instead, there have been what Cuban describes as ‘hybrid’ changes—mixes of teacher and student-centred approaches. Whilst the iTEC approach has been successful with a relatively small cohort of teachers, further work is required to understand if and how learner-centred digital pedagogies can be mainstreamed.
Key finding 6: Teachers became more enthusiastic about their pedagogical practices.
Teachers noted that their practice became more interesting through a shift to student-centred digital pedagogies. Teachers were also motivated through seeing the impact the project was having on their students.
Facilitating iTEC Learning Activities impacted on teachers’ motivation and enthusiasm (see Fig. 9.4).
Fig. 9.4
Perceived positive impact on teacher motiviation. The percentage of teachers (n = 826) in agreement, cycles 1–3
×
Qualitative findings echoed that of the teacher survey data with teachers reporting an increase in their own motivation.
The teacher feels much more motivated. His students are learning with fun and experimenting. Their eagerness gives the teacher a positive energy for his future classes and the teacher is more involved in the projects and effective teaching. (Turkey, case study report, cycle 3)
The adoption of constructivist digital pedagogies can have a positive effect on teacher morale (Baylor and Ritchie 2002). Teachers certainly found this to be the case in iTEC; they enjoyed the opportunity to try out new ideas and increase their use of technology.
Key finding 7: Teachers stated that they used technology more frequently; it was systematically integrated throughout the learning process rather than reserved for research or presentations.
More regular, and increased, use of technology in the classroom was perceived to be new for both teachers and students. In some cases, use of technology in lessons by students per se was seen to be novel. 37 % of students (n = 1293, cycle 5) said that the ‘best thing about iTEC’ was the increased use of technology in the classroom (the most frequent response to an open-ended question).
I’ve had this class only from the beginning of this year and the students have almost never used ICT in school so for them everything was new. (Italy, teacher survey, cycle 5)
While the teachers involved in iTEC had used technology to support student research or presentation work in the past, they started making use of technology to interact and communicate with students; facilitate team working; support design and production tasks; assess work; and encourage students’ self-reflection. This can be attributed to the learning design process, which highlights the need to include digital tools in each Learning Activity, thus ensuring that an embedded digital pedagogy is adopted.
Teachers incorporated a wider range of types of digital tools/services into teaching and learning than they had done previously (most commonly for data capture, accessing information, communication, collaboration, media sharing, media authoring and mobile learning). 60 % of teachers surveyed (cycles 1–3 and 5, n = 1048) indicated that they used digital tools/services that they had not used before. Each set of Learning Activities, presented at the start of each piloting cycle, guided teachers to try new digital tools through general recommendations for types of tools such as social networking sites, blogs and mind-mapping tools. The iTEC project also developed a number of prototype tools. These were introduced to teachers at various points in the project and incorporated into piloting activities by some of them.
Teachers (n = 583, cycle 4–5) were asked to rate how different their use of technology was when implementing the Learning Story, in comparison to what they were doing before, on a scale from 1 (not at all) to 10 (radically different). 30 % of teachers stated that their technology use had changed substantially (a score of 8–10). One in four teachers (26 %) perceived that their technology use was not markedly different to their previous teaching methods (a score of 1–4). Again, this is unsurprising given that many teachers perceived that they had a high level of ICT competence.
The difference between the maths lessons and the other lessons is that in these lessons we work a lot with GeoGebra, with Facebook, and with Glogster and we record things and in other lessons we don’t. In the other lessons the most we can do is some work on the computer once in a while. (Portugal, student interview, cycle 2)
We used technology in every step: pupils searched for all the information about the content from internet, videos, by email or from experts who visited our school. They learned to send emails to experts. They also used iPads for the first time and shot a video and edited the video by using iPads. They reflected their learning using TeamUp tool. (Finland, teacher survey, cycle 4)
There are continued claims about technology’s potential to enhance teaching and learning (OECD 2013). However, as already mentioned above, very few teachers in Europe use technology to support teaching and learning, other than for lesson preparation (EC 2013). Furthermore, student use is still limited, with one in five rarely using digital tools in lesson time, despite infrastructure having improved substantially (EC 2013). The adoption of the iTEC approach by teachers has led to the systematic integration of digital pedagogies in the classroom and increased use by students.
Key finding 8: Teachers collaborated more, both within and beyond their schools, a process facilitated through the online communities.
The iTEC approach led to increased collaboration between teachers. Training and support were positively received by teachers who particularly enjoyed face-to-face meetings, networking with other teachers, opportunities for hands-on experience of tools, online discussion forums, webinars and video-tutorials.
Working with ITEC has motivated me to engage other colleagues. It awakened a strong desire not to deal with this project on my own. The challenge is to untangle the frameworks in which we work. (Israel, teacher focus group, cycle 4)
Another innovation is the development of a community of practice of teachers. Dissemination by teachers has taken place via a national blog and websites. There has been an increase in collaboration and interaction between teachers. (France, national case study)
The use of national online communities was evaluated in cycle 4. Although how the online communities were used varied, they were most commonly used to share ideas and examples of good practice. They were also used to support collaborative problem solving within the online community, but this was a less frequent activity (except in communities expressly intended for this purpose).
Professional networks for teachers will become increasingly important as teachers need to continuously update their practices (Redecker et al. 2011; Johnson et al. 2014). Many teachers recognise the importance of the internet in facilitating such networks (Purcell et al. 2013). Moreover, participating in such online communities fosters a positive attitude to collaboration, sharing resources and supporting peers (Tseng and Kuo 2014). However, only one in three teachers in Europe are at schools that support collaborative approaches to learning design (EC 2013).
What Is the Potential of the iTEC Approach for System-Wide Adoption in Schools?
This section considers the evidence of the potential of the iTEC approach for system-wide adoption.
Key finding 9: The scenario-led design process can support mainstreaming of innovation, providing the process is refined.
Policy makers felt that the iTEC scenario-led design process would be an important output of the project in relation to policy-making and the potential for supporting scale-up of digital pedagogy through professional development.
The scenario development toolkit is seen as a real asset in Hungary…it is seen to facilitate a professional approach to developing and documenting best practice. (Hungary, national case study)
The scenario-led design process, once finalised, also has the potential to be included in initial teacher training programmes and continuous professional development (for school leaders and teachers). For example, the scenario development process has already been integrated into a Masters level programme in Estonia and is considered to fit well with course aims; its use will continue there in future years.
Key finding 10: The library of scenarios, Learning Stories and Learning Activities was viewed by policy makers and teachers as a valuable output of iTEC to support system-wide classroom innovation.
The library of Learning Stories and Learning Activities was perceived by teachers to have the potential to lead to both pedagogical and technological innovation in the classroom (93 %, cycles 1–5, n = 1399). Policy makers noted that the library of resources provides an effective structure; the resources are sufficiently innovative without being overwhelming; and are easy for teachers to use. In addition, they suggested that Learning Activities are valuable because they provide concrete examples of novel approaches, emphasise innovation and flexibility, and encourage teachers to become learning designers. 85 % of teachers (cycles 1–4, n = 1152) said that they would use the Learning Stories they had piloted again whilst 86 % of them said that they would recommend the Learning Story to other teachers.
The iTEC scenarios and Learning Stories provide a good structure for teachers. The scenarios received a lot of attention in Estonia. (Estonia, national case study).
The Learning Activities are valuable because they are very practical and show teachers how a lesson can be structured. The fact that they are concrete examples, rather than general descriptions is valuable. (Czech Republic, national case study)
There needs to be systemic changes and/or incentives if the iTEC approach is to be widely adopted. Within school contexts, a risk-taking culture in relation to the adoption of digital pedagogy should be encouraged (Niemi et al. 2013). There is also a need to develop teacher education such that effective integration of ICT can be modelled and teachers can be encouraged to become agents of change (Twining et al. 2013; Brečko et al. 2014). In common with the literature, the evaluation has provided evidence that an incremental approach to innovation, such as that facilitated through iTEC, can be successful (OECD 2008; Kampylis et al. 2013).
Key finding 11: Awareness of the iTEC approach is growing in educational systems, and there are signs of widespread uptake.
In cycle 5, nine out of ten teachers (n = 244) said that they intended to use the iTEC approach again in the future (91 %) and would recommend it to other teachers (92 %). While 81 % of teachers (n = 244) agreed that the iTEC approach could become part of their own routine practice, only half of them (52 %) agreed that such methods could become part of the routine practice of other teachers in their school. They were particularly cautious about the potential for upscaling at national level with only 43 % agreeing that the iTEC process could become part of routine practice for the majority of teachers in their country.
Yes, it has the potential to change my future practice because now I have learnt about other ways to get my objectives, other ways to work in groups with my students, other ways to do collaborative work, and I’m going to use it in my future lessons (Spain, teacher interview, cycle 4)
Teachers were asked if they had shared their experience of various aspects of the iTEC approach with teachers outside the project (both within and beyond their schools). They indicated that they had shared both the Learning Story they had implemented (83 %, cycle 4, n = 331), and the iTEC approach (86 %, cycle 5, n = 244).
There is some further evidence of transfer of the iTEC approach within schools (cycles 3–5: 13 of 68 case studies), and of other teachers expressing an interest (cycles 3–5: 19 of 68 case studies; cycle 5: 54 % of teachers surveyed, n = 245). Other schools had held, or planned, training events and in many cases head teachers actively supported dissemination (an enabler of transfer). In contrast, there was some evidence of perceptions that other teachers might not be interested in the iTEC approach or would find the use of technology challenging. Similarly, teachers from cycle 5 (n = 244) reported that about one third of teachers they had shared the iTEC approach with had mixed reactions and 14 % were not interested.
They were aware of it; K informs us regularly. She talks about it in e-mails, personal conversations and at meetings. Thus, teachers are aware of it, and are curious to know about the latest project K is involved in. This is how far we got. I think later on other colleagues may join too. (Hungary, head teacher, cycle 3)
But in my school I have introduced quite a lot of ideas. A good example is mathematics, where they are making Learning Stories. There are also teachers who have started to use TeamUp. (Estonia, teacher interview, cycle 4)
Transfer to teachers beyond participating schools was less commonplace, with some indications of reticence to share beyond colleagues due to lack of confidence in technical ability, the challenge of project jargon, and competition with local schools. Nevertheless, there were a small number of examples where this had happened in each cycle. For example, one teacher in cycle 4 had presented their work at a conference for mathematics teachers and in cycle 5, teachers from two countries (Estonia, Lithuania) had spoken about iTEC at national conferences. Others indicated that they believed that dissemination should take place, but this needed to be organised centrally, rather than by individual teachers:
More visibility on expositions and meetings for people working in education. For example at the colloquium for head teachers that is being organised annually. Every school shows what they have achieved in the past year. That is where iTEC should be made visible. (Belgium, teacher interview, cycle 4)
I believe that the research and knowledge-based communities in and around city T’s schools are very interested in being part of something bigger and in disseminating this to a wider audience. At the same time there are 53 primary and lower secondary schools in city T, so it’s clear that sharing with other schools is a challenge. (Norway, head teacher interview, cycle 4)
Key finding 12: In countries in which iTEC aligns closely with current policy direction, the iTEC approach is likely to be adopted and to influence future practices.
The national case studies were undertaken mid-way through the third year of the project, partly focusing on the impact of iTEC on ICT strategy and policy development. Although it was seen as early days, there were initial indications of potential impact in some countries. Dissemination was already taking place in many of the participating countries, with seven indicating that they had held seminars, workshops or forums, and five stating that they had held conferences.
In one country (Norway), iTEC had already been influential and had been referenced in official government consultation papers, whilst in five further countries (Austria, Belgium, Estonia, Finland, France), the iTEC project was noted to align with current policy direction, and was therefore likely to be influential in the future.
…this is the right time for policy recommendations to be included in the National Strategy of Education in Estonia. There is a chapter within this on ‘digital culture in education’. The underlying ideas of iTEC appear to be very similar to those in the National Strategy. (Estonia, national case study)
iTEC correlates quite well with other national developments, including the development of a new core curriculum, and the aim to digitalise the national matriculation exam in a few years. So, iTEC comes at a good time. (Finland, national case study)
All project partners intend to make iTEC outputs available on national portals and/or link to resources that are centrally maintained. Other future plans included: holding closing conference; producing and disseminating national publications; awareness raising events; running further training events for teachers, head teachers and/or ICT coordinators; integrating iTEC with existing online training provision; localising Future Classroom Lab modules; integration with new/ongoing projects; establishing networks of interested initial teacher training institutions (ITTs); running conferences for ITTs; co-ordinating dissemination through one or more ITTs; making initial contacts with ITTs; investigating accreditation options; and maintaining Future Classroom online communities.
Future Classroom Lab modules have already been embedded in Masters programmes and professional development courses. The University of Lisbon, a partner in the iTEC project, has been particularly proactive in bringing together representatives of ITT providers, developing a call for action document to target ITTs and policy makers. Hungary plans to localise the Future Classroom Lab modules for Hungarian teachers and has been closely involved in the preparation of the forthcoming National ICT strategy which highlights innovative learning approaches through digital pedagogies. In Italy, iTEC has become part of the Digital School strategy. Thus, an additional two countries have stated that iTEC has strongly influenced recent national ICT strategy development (Hungary, Italy).
‘Bringing a technology innovation to scale in education requires a design that is flexible enough to be used in a variety of contexts and robust enough to retain effectiveness in settings that lack conditions for its success’ (Clarke and Dede 2009:364). The signs of widespread uptake suggest that the iTEC approach could meet these necessary conditions of flexibility and robustness. However, few ICT innovations in the classroom survive beyond the early adopter stage (Kampylis et al. 2013). Therefore, organisational structures will need to be put in place to support the continued adoption of the iTEC approach. Policy and programme alignment is important for maximising impact (Kozma 2005); more could be done to understand the challenges and requirements in countries where this is not yet the case. Integrating the approach in teacher education will model effective use of ICT as well as the iTEC approach, and encourage teachers to become agents of change (Twining et al. 2013; Brečko et al. 2014).
Conclusions
School staff almost unanimously agree that integrating ICT into learning and teaching is necessary for ensuring students are prepared for the twenty-first century (EC 2013). However, whilst almost all teachers use technology to help them prepare, ICT has not yet become embedded in teaching and learning; use in the classroom is variable (EC 2013).
The project has created a tool kit and professional development resources to provide continued support for the approach; these resources can be (and are being) localised at national level by many of the partners who participated in the project. The main outputs of iTEC are:
a scalable scenario-led design process for developing digital pedagogy;
the Future Classroom Toolkit and accompanying training provision;
an extensive library of Future Classroom Scenarios, Learning Activities and Learning Stories.
The iTEC approach, in the form of a learning design process and the library of resources created through the project, has led to the adoption of digital pedagogies and the increased use of technology in European classrooms. Most teachers were incredibly positive about their experiences of adopting the iTEC approach, plan to use the ideas in the future and have shared their experiences with colleagues. The evaluation evidence suggests that the iTEC approach can further contribute to the continued uptake of digital pedagogy, if the appropriate support systems such as professional development and online communities of practice are put in place.
Lessons Learned
Given the length of the iTEC project (4 years), it was inevitable that priorities changed, both as a result of internal and external drivers. Furthermore, the design of the project with five overlapping cycles of development, piloting and evaluation created challenges. For example, before Cycle 1 had been evaluated, Cycle 2 was at its midpoint and Cycle 3 had already begun. While formative data were shared with relevant stakeholders in iTEC, the extent to which the evaluation findings were able to input into the development of the project was more limited than might otherwise have been the case. Future large-scale pilots should take this into account and ensure that feedback can be informative and useful.
As with most evaluations, resource constraints were an issue. As the project developed, the number of themes and issues which needed to be covered in the evaluation extended, meaning additional tasks had to be added which had not been anticipated at the start of the project. Whilst adopting a responsive, flexible approach, it was not possible to evaluate all aspects of this complex project thoroughly. Whilst impossible to predict changes over time it would be helpful to prempt where possible and identify which aspects should be prioritised. Regular reviews of the aims and scope of the evaluation would be beneficial. A further tension was the need to evaluate both project processes and outcomes. Again, resource constraints meant at certain points, strategic decisions needed to be made about where resources would best be focused and this could be built into a review process from the outset.
The timing of the development of the various iTEC tools and technologies also presented evaluation challenges. The majority of the tools, and the toolkits, were only introduced in the final two cycles. This meant there was limited opportunity to gather data and it was not possible to focus on each tool in the depth which might have been possible had they been introduced into the project more gradually. Future large-scale pilots would benefit from incorporating existing technologies which require minimal refinement alongside the development of new technologies. It would also be helpful to review the management procedures for technology development so that delays can be minimised.
Naturally in a project involving such a large number of countries, language barriers presented challenges to the evaluation. This limited the amount of qualitative data which the evaluation team were able to collect directly. The approach used was to support National Pedagogical Coordinators (NPCs) in conducting qualitative evaluation activities in the national language in each country. This produced highly mixed results; while some countries supplied detailed, high quality case studies, many NPCs clearly struggled with this task, especially in earlier cycles. This situation may have been improved by including research experience as one of the selection criteria for NPCs to ensure they had the necessary skills and knowledge to collect the data required for the evaluation. An alternative approach would have been to redirect resources away from supporting NPCs in data collection and instead for the evaluation team to conduct a smaller number of case studies each cycle (with the help of translators where necessary). This would have resulted in a smaller amount of data, but of considerably higher quality (the key factor with qualitative data).
Another language barrier related to the translation of the open-ended survey responses, completed by teachers in their national language. In cycles 1–4, Google Translate was used, but this was not an adequate solution at the time as it frequently did not provide sufficiently meaningful or in-depth translations to allow the data to be coded accurately. In cycle 5, an alternative approach was adopted: responses from a limited number of countries were translated by hand, providing more accurate translations which it was possible to code with much greater granularity. The remaining data were then transcribed using Google Translate and allocated to the codes generated through analysing the data which were translated manually. This helped to act as a check that the themes emerging from a subset of countries matched those found in the wider data. Whilst focusing on data from a limited number of countries is not a perfect solution, this allowed the open-ended survey responses to be analysed in a more robust way within the resource constraints of the project.
Throughout the project, the need to report data collected from such a diverse range of countries in a meaningful way was a challenge. This was exacerbated by the differences in sample sizes between countries in each cycle. As the project progressed, we became increasingly aware of the need to stress the limits of the data and to report it in ways which were less open to challenge, even though this meant that it was not possible to provide the level of detail others wished for (for example Ministries of Education that participated in the project). Future projects of a similar scale and design would benefit from greater emphasis on qualitative data such as in-depth case studies.
Recommendations and Implications for Practice
The iTEC project has provided evidence that an incremental approach to change, at the heart of the learning design process that was developed, can be effective. The findings, and the evidence behind them gathered during the project, naturally lead to a number of consequential implications that impinge on policy making, learning management, technology provision and research. To conclude this chapter, we now present these recommendations.
Policy Making
Towards a learning culture. Mechanisms and structures should be put in place, supported through changes to formal curricular and assessment systems, to encourage the development in schools of a culture of self- and peer-reflection, continuous development, new roles, innovation and risk-taking, in order for schools to continue to be fit for purpose, to exploit new opportunities, and to meet evolving needs. Such changes should be communicated effectively to all stakeholders, including parents, in order to encourage positive attitudes. The potential of the iTEC approach and legacy resources to support this culture should be exploited in professional development, online communities, and through teacher ambassadors. This is particularly true in countries where the iTEC approach aligns closely with national policies and strategies. Opportunities to incorporate the iTEC approach in initiatives and programmes related to twenty-first century learning and change in schools should be identified.
Investigate learning outcomes. Further, larger-scale, impact studies of classroom implementations of iTEC tools, Learning Activities and Learning Stories at national level (including randomised controlled trials) could be commissioned, focusing on learning outcomes (specifically twenty-first century skills) and student attainment. The revised Future Classroom Toolkit could be validated in countries where the toolkit clearly supports current policy directions.
Build teacher capacity. Policies and support systems, including professional development, technical and pedagogical support, should be put in place to (a) develop teachers’ digital competence, particularly in digital pedagogy, and (b) facilitate teachers’ engagement in collaborative processes for learning design. Cost-effective online professional development, such as MOOCs and communities of practice, should be supported at national and international level, including the use of video clips and screencasts to enable teachers to share ideas and good practice. The potential for integrating iTEC assets (the Future Classroom Toolkit, Scenarios, Learning Activities and Learning Stories) created within national professional development structures and initial teacher training should be explored further. To facilitate this trainers and teacher educators would benefit from targeted development on the use of the toolkit and should be supported to use the toolkit in their own practice.
Management of Teaching and Learning
A culture of collaboration. School leaders should put in place organisational structures (e.g. embedding professional network participation in the school culture, and ensuring that teachers have sufficient time for effective networking) and incentive schemes to ensure that teachers share their experiences with other teachers, within and beyond their own school and develop positive attitudes towards teacher networking and collaboration. Teachers should establish and maintain connections with colleagues in their own school, and beyond, to share and jointly develop digital and pedagogical knowledge and skills as a community.
Twenty-first century competencies. Teachers, supported by school leaders and through professional development, should create opportunities for students to take greater responsibility for their learning, work collaboratively, engage in authentic learning experiences and develop twenty-first century skills through the adoption of digital pedagogy. This demands a shift in teacher and learner roles. It also demands a positive attitude towards change, innovation and risk-taking. As students engage in more active and student-centred learning approaches, the development of digital competence becomes increasingly important.
Technology Provision
End-user involvement. Technology providers should take account of the lessons learned through the iTEC project in relation to meeting needs, evolving pedagogical practices, motivating and engaging teachers as partners rather than end-users in product development and testing.
Product development. Of the various iTEC prototype technologies developed, the Scenario Development Environment would benefit most from further research and development with a view to its commercial development. It would be beneficial to conduct a larger scale pilot study, particularly in the countries where it was received favourably.
Research
Research topics. Research should continue to study whole school change, new ways of designing and managing learning, and pedagogies that make most effective use of new digital tools to produce desired learning outcomes. Research should build on iTEC results and investigate further how best to mainstream technical and pedagogical innovation, assessing both radical and incremental approaches in school education contexts.
National specificities. Further research should be undertaken in countries in which the iTEC approach does not align so closely with national policies and strategies to identify how the approach could be adapted to fit different needs.
Research methodology. It would be beneficial to analyse, refine and validate methodologies for large-scale evaluations of projects lasting more than 2 years, where the object of study and the technologies used themselves evolve. Developing approaches for assessing learning outcomes in such conditions would be worthwhile.
Acknowledgements
We would like to acknowledge the invaluable contributions of everyone involved in the evaluation activities in the iTEC project including: Maureen Haldane who co-lead the work initially; Roger Blamire who played a major role in shaping the final evaluation report; Will Ellis for his excellent project management skills; the National Coordinators who went to great lengths (collecting data, writing case studies, translating research instruments) to support our work; all the teachers and students involved in piloting activities; and finally our colleagues from Manchester Metropolitan University—Geoff Bright, Helen Manchester, Jonathan Savage, John Schostak, Nicola Whitton, Charmian Wilby, and Adam Wood.
Open Access This chapter is distributed under the terms of the Creative Commons Attribution Noncommercial License, which permits any noncommercial use, distribution, and reproduction in any medium, provided the original author(s) and source are credited.
