The Patterns of School Improvement in Digitally Innovative Schools

As the main goal, school improvement is aimed at improving students’ learning outcomes (Creemers & Reezigt, 2005). Schools’ digital transformation may be done by restructuring teaching and learning practices, re-envisioning learning spaces (Sheninger, 2014), leadership practices (Sheninger, 2014; Zhong, 2017) and the ways pedagogical approaches are organised (Crook et al., 2010; OECD, 2015). The implementation of digital technology into educational practices contributes to student learning (Wong & Li, 2011) and educator capacity (Haynes & Shelton, 2018). At system level, a school has to improve itself as a dynamically responsive sustainable learning organization (Senge et al., 1994) that is able to maintain the school improvement towards innovative change (Jeladze & Pata, 2018). Researchers have defined some key characteristics of a learning organization: mutual trust and willingness to engage in open communication by the participants (Creemers & Reezigt, 2005; Senge et al., 1994); teachers’ shared values with a focus on student learning (Leclerc et al., 2012); and collaborative knowledge-sharing as a tool for constant growth of both teachers and schools (Fullan, 2001). Tam et al. (2018) explain school improvement through Hargreaves’ (2001) intellectual, social and organizational capital components. Intellectual capital relates school personel’s knowledge, skills, values and dispositions towards ICT with school level goals embedded in curricula, rules, agendas etc. Social capital is more influential than intellectual capital and it comprises of trust, mutual respect and reciprocity among teachers, students and parents as well as communication channels and networks among them. Social capital is considered the lead strategy in enacting change and improving teaching at schools. Organizational capital associates with school leadership practices, the knowledge and skills at change management level to efficiently make use of and guide the intellectual and social capital. Scheninger (2014) suggested that digital leadership requires a shift in leadership style from one of mandates, directives, and buy-in, to one grounded in empowerment, support, and embracement as keys to sustainable change. According to him, digital leadership can be defined as establishing direction, influencing others, and initiating sustainable change through the access of information, and establishing relationships.

To successfully inspire and lead schools’ digital transformation and meet the requirement of visionary leadership, awareness of digital management and support from all stakeholders are two important factors that school principals need to be equipped with (Zhong, 2017). The hierarchical leadership models or distributed leadership models (Spillane, 2006) may be applied and sustained by sociocultural and institutional norms. Distributed leadership model has been found to coincide with project-based approaches in schools (Levin & Schrum, 2012) and the increase of organizational capacity for ICT integration (Tam et al., 2018). Tondeur et al. (2008) created a conceptual model of school-level factors that can contribute to efficient ICT use for schools’ holistic improvement. These factors include the provision of appropriate ICT infrastructure, development of positive teachers’ and leaders’ attitudes towards ICT, development of teachers’ ICT skills, use of ICT in learning and teaching.

School’s capacity could be improved by using ‘guiding instruments’ like D-LIFE (Haynes & Shelton, 2018) and ‘monitoring instruments’ such as SELFIE (Kampylis et al., 2016) or Digital Mirror (see below) that contain indicators of digital maturity. For example, D-LIFE guide addresses the digital infrastructure and resources component with stabilizing and future-directed sustainability notions; learning and support component to raise educators’ and students’ intellectual capacity; longitudinal evidence-based feedback loop component from learning and practices’ data, and sustainable and proactive policies’ component. Although the D-LIFE guide incorporates some of the important elements defining digital transformation, it lacks a comprehensive view on schools, in which for instance organisational capital at change management level plays an important role for guiding digital transformation. SELFIE instrument contains three self-evaluation questionnaires for leaders, teachers and students and generates self-reports for schools providing the level of specific digital development indicators: teaching and learning, professional development, content and curricula, assessment practices, leadership and governance practices, collaboration and networking, and infrastructure. In Estonia the Digital Mirror instrument that provides roadmap on three indicator groups (teaching and learning, change management and infrastructure and services) was nationally implemented for supporting self-evaluations on digital maturity of schools (see below).

We build the theoretical frame of our paper on the model of self-organization processes in schools (Jeladze & Patta, 2019), which is developed based on the European digital maturity SELFIE study pilot data. This model depicts schools as holistic digital learning ecosystems where different types of components are systemically interconnected. The self-organization model depicts three loops in a digitally enhanced learning ecosystem: (1) digital learning loop represents the interaction of classroom level components—digital teaching strategies, learning practices and assessment approaches; (2) mediating loop represents digital infrastructure and resources and information provided from socio-technical landscape to which human agents interact; (3) transformative loop combines the components with a transforming agency that are in place on a school level, such as norms and agendas, support, training, motivation management, analytics and evidence-based decision-making etc. (Jeladze & Pata, 2019). The self-organization model highlights the need for cyclical information exchange between digital learning, mediating and transformative loops in school improvement. The strength of the model of self-organisation processes in schools is that it takes the holistic view on schools as digital learning ecosystems with a number of interconnected variables. We explore in this study the variables from digital maturity self-evaluation instrument Digital Mirror by taking the systemic approach of holistic learning ecosytems as suggested in self-organization model.

Maturity concept has been used for evaluating schools and it relates with relative states of digital innovation. The maturity concept is adopted from natural ecosystems, which develop successively from early to mature stages (Odum, 1969; Chorley et al., 1971). The digital maturity evaluation tools enable describing the static states of the learning ecosystems at certain time points. Digital maturity evaluation suggests that a number of successive stages of digital innovation happen in schools. Several frameworks and tools have been developed during the last decade to evaluate different aspects of schools’ digital maturity: whole-school’s use of ICT and digital pedagogical methods (OPEKA, eLEMER, SELFIE), leadership and governance for the change practices (Microsoft leadership transformation self-reflection Tool), school’s potential in ICT (Ae-MoYS), ICT-enabled innovations in different learning settings and implementation strategies, and the effectiveness of learning (Giovanella, 2016; Galego et al., 2016). Some tools (eLearning roadmap, Future classroom Maturity Modelling, Digital Mirror) provide the roadmap to expand further the innovative technology-enhanced practices in schools by postulating the development stages. These evaluation frameworks examine common educational dimensions for depicting technology use. They explore pedagogical methods that are enhanced by technology, change management and existing infrastructure and access to it. Sergis and Sampson (2014) proposed in schools’ ICT competence profiling framework the same factors for school profiling and added the level of ICT use in curriculum and the ways ICT is used in daily teaching and learning practices. Many of these frameworks monitor only internal factors without exploring the interrelations and interdependences of thise factors and do not embed the schools into the regional settings or explore the interaction between schools. Some researchers (Giovanella, 2016; Galego et al., 2016) have also evaluated external components as part of digital maturity framework. Sergis et al. (2018) developed the concept of school analytics—school-level educational data that inform schools’ strategic decision-making. Previously described frameworks tackle important aspects that define and guide digital maturity of schools, however, they fail to recognize that different components of the system are coupled together and may influence each others’ functioning. Also Sergis and Sampson (2016) emphasize the need for a holistic method that supports analysis on three different layers in order to have an explicit view of school system behavior: (a) microlayer of learning and assessment practices in school; (b) meso layer of monitoring and assessment of teaching practices in school; (c) macro level—orchestration of school development as an organization (Sergis & Sampson, 2016). In this paper we specifically target the schools’ digital maturity as systemic level patterns and search for how different maturity indicators contribute to it.

Digital transformation takes place step-by-step because not all the resources and knowledge for a workable solution are available (Teiniker & Seuchter, 2018). Also, organisations are facing many challenges during the process of digital transformation. Based on a literature review of digitalization in adult and continuing education, Bernhard-Skala (2019) found three main challenges in digital transformation: information technology-infrastructure, staff development and management/leadership.

In the study of Jeladze and Pata (2019) several digital maturity states of schools were described by using the European’ SELFIE pilot study data and the created paths models between the studied variables. Here we briefly describe some successive digital maturity school types from that study that we later intend to compare with our findings in the Estonian context where different set of indicators, but similar indicator groups were used. Digital teaching strategies centered schools had digital strategies in place that might have supported digital activities but teachers were seldom involved in designing school vision and agenda. These schools lacked digital infrastructure and support mechanisms to digital practices and digital learning activities were not very frequent. There were no loops between learning at classroom level, infrastructure and change management components. Digital infrastructure-centered schools had medium level digital infrastructures, but the infrastructure development was weakly connected with change management. Support mechanisms were directed to digital infrastructure but not towards students’ digital learning or digital teaching strategies. The loops were weak between classroom level digital teaching and learning practices and change management for digital change. However, digital practices, learning and assessment were well interconnected. Organizational learning-driven schools were digitally most mature and had strong loops between the interconnected teaching and learning, infrastructure and change management components, yet the evidence-based decision-making loop was weak.

In the current study, we aimed at validating these digital maturity states found based on European schools’ pilot dataset of SELFIE with the different dataset collected from Estonian schools using the officially collected data from Digital Mirror instrument (https://​digipeegel.​ee/​) funded by Estonian Ministry of Education and Research.

Since 1997 dedicated programmes and foundations for supporting the digital turn in schools have been initiated in Estonia. There is a drive from the industry as well as from the government to make education more digital and innovative. At first the Tiger Leap Foundation focused on connecting schools, encouraging teachers to develop their own materials and creating their own training courses. Also, the Tiger Leap Foundation provided teachers with laptops. With respect to technology, the local authority in Estonia is obliged to maintain the schools’ technical equipment and network connections as well as provide tools and services that help to improve education. Additionally, IT companies, such as Microsoft and Samsung, in collaboration with university trainers and researchers, have contributed to the digital turn in schools by running several projects. School team training has been used to empower digital transformation (Lorenz et al., 2016). The Estonian Ministry of Education and Research Digital Turn Programme 2015–2018 pointed out three directions: digital culture integration with teaching and learning; development and accessibility of digital teaching materials, and equipping the schools (network and technology). The Estonian Lifelong Learning Strategy 2020 has provided guidelines for schools to implement digital turn in terms of digital learning resources, digital infrastructure for learning, development of digital competences, changed teaching and learning practices. The new Strategy for 2035 sees the main challenge in developing learning analytics enhanced personalized learning paths. The Education and Youth Administration (harno.ee) is responsible for envisioning, developing and maintaining national components of digital learning ecosystem (such as e-learning environments, e-schoolbag, digital learning resource repository, e-diaries, digital assessment systems) and digital infrastructures and network services in schools. It leads the digital skills development providing digital competence frameworks aligned to DigComp 2.0 and DigCompEdu for students and teachers (including syllaby and job accreditation frameworks), and digital maturity requirements for schools. Free trainings to teachers and school leaders are provided nationally as well as by higher education institutions. The national curricula have required integrating digital technology use at schools in all subject lessons. The educational technologists have been trained since 2009 and they have mediated digital transformation in schools and promoted technology-enhanced learning at organizational level.

The data from schools’ digital maturity evaluation in 2016/2017, 2017/2018 and 2018/2019 with Digital Mirror were used for the analysis. During these years 92 schools evaluated their digital maturity only once, 363 schools twice and 91 schools three times. The summative sample of 962 schools (including 499 different schools) was used for cluster and pattern analysis. Using Digital Mirror for evaluating schools’ digital maturity was set as a precondition for the schools to receive funding for digital innovations from regional budgets. This sample covers 81% of all schools in the country and is representative to the educational level and language distribution of schools in Estonia.

School teams participating in the study filled in the self-assessment instrument Digital Mirror to assess digital maturity of their organisation. Self-assessment was organised by the school teams—in the first phase the team of teachers and educational technologyst/IT manager filled in the survey based on the framework in Digital Mirror (see details of these below) (Fig. 1) collaboratively. Digital Mirror requires also adding evidences as documents to support self-assessemnt. In this study we do not study these evidences. In the second phase school leaders re-assessed or confirmed the assessment results together with the school team, set the roadmap for the next period and submitted the self-assessment results in Digital Mirror. The Digital Mirror framework describes the digitally enhanced school from three dimensions: Teaching and Learning, Change Management and Infrastructure and Services.

The Digital Mirror instrument comprises of 15 self-evaluation variables (see Table 1 below) that were scaled on 5 levels:

The items for assessing digital maturity on the 5 level scale are presented in the Table 1 and for each item, school team had to also provide evidences to demonstrate their level of maturity. Links and descriptions of evidences were presented in the Digital Mirror. All these self-evaluation variables and their explanations were also provided to the school teams.

The results of the internal consistency test on all variables was 0.925 and for components ‘Teaching and Learning’ 0.82, ‘Change management’ 0.86 and ‘Infrastructure and services’ 0.84.

We analyzed jointly the dataset from two periods 2016/17 and 2018/19 to discover the patterns of organizational change management in digitaly enhanced schools. This dataset comprised of two measurments from each school, but we assumed that a larger dataset would enable us to detect clearer patterns.

The following statistical analyses with IBM SPSS Statistics software were conducted. Firstly, hierarchical clustering was conducted and the results predicted 3–4 clusters. Secondly, k-means cluster analysis was run with 3 and 4 clusters to identify the digital maturity clusters of the schools. The 3-cluster model was selected based on the analysis. Next, testing the mean differences among clusters was performed to extract the most significant indicators of digital maturity clusters. Then, canonical discriminant analysis using the “Enter the independents together” method was performed to describe the principal factors informing the clusters in selected cluster models. Last but not least, principal component analysis (PCA) was conducted separately with each cluster data to identify the interrelations among the variables in each cluster.

K-means cluster analysis differentiated 3 digital maturity clusters where the levels of all digital maturity variables differed significantly (p < 0.001): Cluster 1 (C1)—181 schools, Cluster 2 (C2)—357 schools, and Cluster 3 (C3)—424 schools (Table 2). We particularly observed, that Learning Organization variable was among highest in C1, while in C2 and C3 it was assessed of being on a relatively low level.

For looking the specific variables that contributed to the grouping of schools into different school types we performed canonical discriminant analysis. The discriminant analysis detected two functions (see Fig. 2): Function 1 (Df1) described 97.7% of the variance (λ = 0.168, χ2 = 1699.86, df = 30, p < 0.001), and Function 2 (Df2) only 2.3% of the variance (λ = 0.905, χ2 = 95.024, df = 14, p < 0.001).

Df1 correlated positively with the ‘Learning and teaching’ component variables (Digital competence (r=0.414), Teacher’s role (r=0.438), Structural change (r=0.410), Learner’s role (r=0.347), Digital practices (r=0.291)), ‘Change management variables’ (Participatory management (r=0.456), Learning organization (r=0.401), Strategic planning (r=0.428), Leadership and Analytics (r=0.366), Leadership (r=0.372)) and ‘Infrastructure and services’ component variables (IT-management (r=0.443), Services (r=0.398), Devices (r=0.348), User Support (r=0.325)). Df2 correlated negatively with IT management (r=−0.591) and Services (r=−0.466), and positively with Structural change (r=0.409) and Teacher’s role (r=0.323).

The clusters could be separated by Df1 into three progressive stages of schools’ digital maturity, indicating that the most influential variables according to function coefficents were: Teacher’s role (0.332), Digital competences (0.253), Participatory management (0.236) and the IT management (0.253).

Df2 differentiated C3 cluster as being more than C1 and C2 oriented on the Participatory management (0.383), Structural change (0.353) and Teachers’ role (0.202), and less on IT-management (−0.493), Services (−0.380) and Leadership (−0.299).

We performed principal component analysis (PCA) separately for each cluster variables to take an inside look into how the variables that could illustrate the school improvement patterns associated with each other.

The results of Bartlett test of spherity were following:

Four factors were differentiated in Cluster 1 with PCA (see Table 3). The cluster included schools that are digital transformation led and have inclusive and evidence based change management practices.

Identified patterns in Cluster 1 were following:

The IT-management, Leadership role, Digital competences, Learning Organization and Teachers’ role are the interconnecting variables between the factor components in this cluster.

This digitally most mature Cluster 1 is similar with the Organizational learning-driven schools’ cluster description found in the European sample (Jeladze & Pata, 2019) (see above). Based on that study, in this cluster the self-organization loops have emerged between ‘Learning and teaching’, ‘Infrastructure’ and ‘Change management’ components.

Four factors were also differentiated in Cluster 2 (see Table 4). This cluster included schools where digital transformation is appearing only at some practices and these are disconnected from driving organizational goals.

Identified patterns in Cluster 2 were following:

The Learning organization and IT-management are connecting variables between factors in this cluster.

We may compare our least digitally mature Cluster 2 with the Digital teaching strategies centered schools cluster found in the study of Jeladze and Pata (2019). Appearently the digital practices brought in by innovator-teachers is the first stage in schools’ digital transformation. Evidences show that the request from teachers-innovators as well as educational technologists may promote the digital transformation.

Five factors were differentiated in Cluster 3 (see Table 5). This cluster included schools that are in transition showing that digital transformation has started in the structural changes level but IT management practices are not participative.

Identified patterns in Cluster 3 were following:

The Participatory management, Structural change, Network and IT-management are variables connecting the factors in this cluster.

The medium level digitally mature Cluster 3 in our study resembles the Digital infrastructure centred schools in the study of Jeladze and Pata (2019), particularly with the separation of IT management from other change management processes.

We found three stages of digital transformation in schools:

Taking a holistic system view on schools’ digital transformation enables to examine school profiles, their stages of digital transformation as well as understand the interplay between different variables. Being aware of the level of digital maturity at certain time point and based on that define its limitations and growth potential, a school can make development plans and improve itself as a dynamically responsive sustainable learning organization. Furthermore, such digital maturity profiles are needed on a national policy level to plan school improvement activities and programs, allowing to target the variables that need most attention, to support schools moving to the next level of digital maturity. Earlier research has demonstrated the importance of change management and school-level knowledge practices in school improvement (Antinluoma et al., 2018) because deliberate efforts are needed to develop high-level pedagogical practices and enhance the digital transformation.

We found that several variables linked different factors in school clusters and the same variables were also defining the clusters. We consider these variables as ‘catalysts’ of digital transformation in schools:

‘Teachers’ role’—teachers’ role is changing in digitally transformed schools and they start providing help to other teachers. Often this is a gradual growth of digitally innovative teachers towards taking informally or formally the role of educational technologist in the school. This finding relates to educator capacity development as a result of digital technology introduction in schools described by Haynes & Shelton (2018).

‘Digital competence’—the digital competences are not taught separately but become an intervowen and invisible part of learning competences which will be developed as part of active learning practices by every subject teacher. It coincides with the digital technology and learning related results by Wong & Li (2011).

‘Structural change’—the changes in learners’ and teachers’ roles towards more active learning and facilitation models bring along the structural changes in the curriculum, time management, classroom settings, extention of the learning spaces, usage and authorship of the learning resources. Sheninger (2014) has suggested the re-envisioning the learning environment as an important digital transformation component.

‘Leadership’—it is the change management instrument, that can trigger through effective motivation management means the development of the learning organization where teachers, school management and IT management could share practices and learn from each other. Sheninger (2014) and Zhong (2017) have found the visionary and all-inculsive leadership important in schools’ digital transformation. Tam et al. (2018) and Spillane (2006) highlighted the role of distributed leadership supported by organizational sociocultural and institutional norms as the suitable form of organizational capital that promotes school improvement. Tam et al. (2018) relates distributed leadership to the increase of organizational capacity of ICT integration. We could observe in our digitally most mature schools’ Cluster 1 that leadership, and possibly the teachers’ and learners’ demand for using digital devices more intensively in the subject lessons has promoted the IT-management for improving networks and services development.

‘Learning organization’—it is the active mutual learning attitude promoted by management with incentives that transforms the teachers to ‘explorative teachers’ who make pedagogical innovations by developing themselves, uptaking from other teachers, accommodating and and testing out new practices, collecting systemically feedback and reflecting to themselves, to learners and to colleagues about the valueable findings that should be widely applied in the school. Similarly, Leclerc et al. (2012) have found that teachers are creating shared values upon students’ learning. The sharing of new digital practices among teachers is driven by the proactive educational technologist who maintains organised regular learning circles in the school, and partnering and network events among the schools. In the study of Tam et al. (2018) the lead innovative teachers played similar role in Hongkong schools that were effectively digitally transformed.

Differently from our expectation, ‘Digital practices’ variable was not among the variables connecting the factors, nor was it one of the school-cluster defining variables. We may argue that active learning practices with digital tools like co-creative, project-based and inquiry-based approaches have not yet transformed the Estonian schools to the new level that requires structural changes. We forsee that there is a potential that ‘Digital practices’ will start to play the leading role when school is able to pick up new ‘Change management’ approaches, and make structural and infrastructural changes as described above. It is notable, that the ‘Digital infrastructure’ variables like ‘Devices’ and ‘Services’ were not the drivers of digital innovation in current Estonian schools. We discovered that in the current period of digital transformation in Estonian schools, the ‘Analytics’ variable appeared not to be among the ‘catalysts’ of transforming the organizations. However, we predict that organizational changes must be evidence based, and in the future ‘Analytics’ such as from actual classroom practices, digital resource usage monitoring, competence gap monitoring, whole school digital maturity monitoring will be one of the variables that closes the loop of organizational learning and speeds up the change.

To summarise, our approach to assess schools’ digital maturity with Digital Mirror provides an evidence-based view on schools’ current situation and helps them to make plans for whole-school digital transformation step-by-step. Overview of these aforementioned ‘catalyst’ variables helps school management as well as ministries to direct their attention, efforts and resources to the most critical aspects and plan supporting activities accordingly. To become digitally innovative, schools have to promote the development of the catalysts indicators as the key drivers of digital innovations in schools.