Technology enhancing mathematics learning behaviours: Shifting learning goals from “producing the right answer” to “understanding how to address current and future mathematical challenges”

This paper considers the challenge and outcomes of creating software to support mathematical thinking behaviours through learning activities in school, to connect across different environments, and provide teachers with necessary information about pupils’ thinking and their knowledge levels. The features of the software created, pedagogic innovations and purposes of the educational software that were developed, mathematics learning arising, and analysis of the attitudes and outcomes of pupils and teachers who used the created web platform, are presented here.

An overview of the context of teachers and pupils in which the system was developed (in Serbia) will initially be presented, together with a discussion and consideration of key factors concerned with the importance of homework, its role in educational systems and the ways it fits into teaching practices. Following the statement of the research questions, problems and barriers that can affect processes of implementation of educational software in teaching are discussed in the literature review, which also highlights key issues that need to be addressed to ensure appropriate use of information and communications technology (ICT) materials in educational processes. Following an overview of the theoretical framework, the design approach to the development of the technologically-based platform called eZbirka, created at the Faculty of Mathematics at the University of Belgrade, is detailed. The aim of the software was to support teachers in: (1) addressing negative behaviours – the copying of homework solutions between pupils; and (2) enabling teachers to be aware of the problems and barriers pupils face when confronted with the need for understanding of mathematical concepts. The research methods for the study are detailed, followed by a presentation of results obtained, analyses of research findings, and discussion of outcomes arising.

As an introduction to the rationale for this study, according to early findings by Hart and Walker (1993) and Wong (2001), the way educational software impacts on the attitudes of pupils and teachers depends on the subject topic for which the software is used, on characteristics of classroom instruction, types of assessment, and material delivery tools. This paper will show how these (and other) factors were considered in the software design approach that was taken. In recent years, education, and especially mathematics education, has had access through research to an increasingly diverse number of different class teaching instructional approaches (National Centre for Excellence in the Teaching of Mathematics 2012; Pelgrum and Law 2009). However, as stated in these sources, the variety of learning instruction often depends on ICT resource availability and support materials for different teaching practices.

In Serbia, there is no organised production of digital materials specifically adapted to the national curriculum or to school activities. Teachers in schools often use data projectors to show Microsoft (MS) PowerPoint presentations, which they have previously created at home (The Institute for the Advancement of Education 2014a). Additionally, they sometimes use blogs to exchange materials with pupils. However, all these activities are based on the individual efforts of teachers (Džigurski et al. 2013).

The national curriculum specified by The Institute for the Advancement of Education of Serbia (2014b) implies that teachers should give homework to pupils consisting of shorter tasks on a regular basis, in order to establish and mark levels of knowledge acquired in school. At the beginning of each class, together with the teacher, pupils report about the tasks and any problems they have had, and if necessary, they re-solve these tasks. Regarding homework, published teaching method instructions given in the guidelines for the implementation of the mathematics curriculum for primary schools state:

“Homework assignments are an important component of the teaching process. They not only test how much the pupils have mastered certain materials, but they present an introduction of independent work and self-education to pupils. Tasks should be varied, and by difficulty should be balanced, in accordance with the knowledge and skills of all pupils.” (The Institute for the Advancement of Education 2014b, p. 83)

In addition to regular classes, which are compulsory for all pupils, regulations specify the need for extra lessons and mathematics classes (for pupils who achieve above-average results) and additional classes (for pupils who score below-average results). Achievements of pupils at school and the work they complete at home help teachers to identify any issues or problems which require more time in additional classes, as well as how to continue the planning of regular curricular activities. The results of homework tasks help teachers to detect problems at the earliest stage that they appear, and help them to address these as soon as possible. Andrić (2011), in a study about homework tasks, makes a comment on their value in the educational system in Serbia, saying:

“Homework assignments are probably the most frequent form of checking pupils’ knowledge and skills in mathematics. They are mainly given to pupils after each teaching period i.e. class, regardless of whether it is time to acquire new knowledge or time for practicing. Homework has an aim to broaden pupils’ individual work and make them rehearse extra in courses and at home.” (Andrić 2011, p. 17)

Reports from teachers in Serbia, however, present another view of homework practices; pupils copy solutions of homework tasks before the class, so they do not do the tasks by themselves at home. This behaviour is reported by teachers to be becoming more common, and large numbers of pupils are finding an easier way to solve homework tasks by simply using copy-paste and write actions. As a result, teachers do not have access to the real level of knowledge of their pupils after each teaching unit. Moreover, teachers do not have a valid instrument that allows them to perceive the problems that pupils experience when completing tasks and hence are not able to plan appropriate ways to address barriers in the knowledge of their pupils. As a consequence, pupils do not understand all the material; they disengage from completing tasks, they do not learn regularly, and they become used to copying. Thus, one very important part of the teaching process becomes meaningless and useless. It is not surprising that pupils continue having difficulties with managing their homework. In such a dysfunctional practice, problems continue to accrue, with less and less chance for solving them.

Indeed, reports from teachers indicate that they become aware of the real problems of limited pupil knowledge and the obstacles they face only after completing regular tests and examinations (one per month). At this later time, teachers report that it is too late to address issues and misunderstandings. In such a system, as stated by Killoran (2003), ensuring pupils complete their homework has become one of the most frequent and frustrating behavioural problems for educators. This was the key purpose for the development of the educational software eZbirka.

The key research questions asked in the study reported here were:

Interactive teaching materials have been found from earlier studies to have the potential of making a significant contribution to the teaching and learning of mathematics (Balacheff and Kaput 1996; Sutherland 2007). For example, Ruthven et al. (2004) in their research stated that when pupils were working on computers and learning with the help of a systematic medium, they were more able to focus on patterns and connections between multiple representations. It was reported in another study that computer use might enable a deeper, more direct mathematical experience (Balacheff and Kaput 1996). Other researchers have pointed out major ways in which ICT can provide better environments for the learning of mathematics. For example: providing fast and reliable feedback (Becta 2009); creating environments with real examples when exploring mathematical problems (Noss and Hoyles 1996); seeing connections between representations and changes in mathematical objects (Condie and Munro 2007); working with dynamic images; and enabling work with real data to be represented in a variety of ways (Godwin and Sutherland 2004).

From a pedagogic perspective, Sinclair and Jackiw (2005), in their study about the features and characteristics of what they called a new wave of ICT technologies in the classroom, identified important pedagogic features that need to be considered when using them. They identified relationships among individual learners; grouping of learners; the role of the teacher; the layout of the classroom; alternative classroom practices; and relationships to the world outside the classroom (Sinclair and Jackiw 2005; Walker et al. 2012). However, although there can be a positive impact of ICT on mathematics teaching, the integration of digital resources into teaching practice is not necessarily straightforward; it seems from a range of studies to be more difficult than expected. Indeed, many studies have shown obstacles that teachers experience when integrating ICT in their classrooms (Beaudin 2002; Jones 2004; Rogers 2000; Snoeyink and Ertmer 2002; Takači et al. 2017). Barriers can be grouped into: lack of confidence among teachers; lack of quality software; poor curriculum connections; lack of time for integration; lack of effective training; technical problems while the software is in use; lack of vision as to how to integrate ICT in instruction; lack of personal access during lesson preparation; and negative teacher attitudes towards computers (Fethi and Inan 2010; Hew and Hara 2007). These factors clearly need to be considered in a design approach for new educational software.

In terms of the influence of design approaches on teacher use, Miller et al. (2008) in their report highlight the problem of appropriate technology choices for education. They state that the key issue for success of educational technology is appropriate choice of software, which will support learning and teaching. An earlier report by Ofsted (the inspection system for schools in England) “ICT in Schools” (2002), indicated that: “Less successful use of ICT in subject teaching core typically stems from weak links between the computer task and the lesson objectives. Too many teachers select software packages for their visual appeal rather than their relevance to lessons.” It is essential, therefore, that the design of educational software should be adaptable to the curriculum for which it is intended. Research studies suggest that beliefs about adaptation, like perceived usefulness of ICT solutions, has a direct influence on systemic use and technology adoption (Venkatesh and Davis 2000). A later study stated that functionality, features and the visual identity of a software solution must be adaptable to pupils’ ages (Farmer and Tilton 2006), and use in learning should be simple, flexible and purposefully focused for pupils and by teachers.

It has been argued that methods and effective ways of implementation through which information technology can improve educational curricular activities have in part been identified but are still yet to be fully found (Romero et al. 2007; Topper and Lancaster 2013). For implementation to improve educational activities, research studies show that it is necessary to plan and develop elements of technology enhanced education that accord with school community, society, present teaching methods and environmental features in which the educational software will be implemented (Hijon and Carlos 2006; Nistor et al. 2013; Passey 2000; Takači et al. 2017). Additionally, studies highlight the fact that implementation should be customised to support specific pupil needs and address observed problems in teaching (Passey 2013; Tapscott 2009). In terms of pupil needs, Clark et al. (2014) investigated factors influencing teachers’ beliefs about their pupils’ mathematical understandings and attitudes, and concluded that teachers could better meet pupils’ needs when they understood pupils’ prior mathematical knowledge, experience and attitudes. Campbell et al. (2014) also highlighted that teachers claimed both awareness of their pupils’ attitudes towards mathematics and their organising of instruction as being important factors to support further progress and proficiency of mathematics skills. In terms of pedagogical interactions, Goos (2010) stated that:

“Opportunities arise at three levels that represent the teacher’s thinking about: the tasks they will set their students (using technology to improve speed, accuracy, access to a variety of mathematical representations); classroom interactions (using technology to improve the display of mathematical solution processes and support students’ collaborative work); the subject (using technology to support new goals or teaching methods for a mathematics course).” (Goos 2010, p. 68)

Recently, questions about how ICT might support education reform in mathematics has focused on exploring educational technology as a tool for development of pupils’ reasoning through problem solving and discourse (Kitchen and Berk 2016). These authors stress that mathematics is best learned when pupils and teachers have the opportunity to engage in brainstorming through which mathematical ideas are generated, shared, and gaps in mathematical understanding are investigated and debated. They highlight the need for research on computer software programs that help pupils to apply mathematics concepts and skills. Foster et al. (2016), for example, evaluated effects of a mathematics software program, ‘Building Blocks’, on young children’s mathematics performance. They concluded that when information technologies are used with educational purpose and to meet pedagogical goals, when there is a clearly defined method of implementation, with a strong continued and planned focus, then positive results are likely to arise. Their findings are commensurate with results from research into computer-assisted instruction (CAI) and from meta-analyses suggesting that mathematics learning of kindergarten-aged children can be enhanced by using research-based software interventions. While the need to use software to focus on the pedagogical context as a whole can bring benefits, the authors stated in conclusion also that, “it remains possible that other supplemental mathematics programs, tutoring, and even mathematics homework could be equally beneficial” (Foster et al. 2016, p. 21).

Relating the concerns and features identified in the review above to homework activities (in mathematics or other subject education), as Cooper (2001) stated, homework is a “battlefield”. Yet homework practice is an issue of everyday importance for teachers, pupils, and parents. Although previous research about homework practice has indicated that forms and qualities of homework can make a difference to pupil knowledge, the connection between work at home and pupil success in school is far from clear. One research study has shown that benefit arising from homework depends on the attitudes of pupils, but also depends on the items, models, frequencies and the form of tasks assigned (Cooper 2007). Homework may also have non-academic benefits; for example, it could affect the self-esteem of pupils, development of work skills, responsibility and self-control (Hong and Milgram 2000). Nevertheless, it has been found that too much homework may diminish its effectiveness and have negative impacts on pupils. Furthermore, Kohn (2006) and Moorman and Haller (2006) stated that homework does not provide any academic benefits for pupils who do not possess the skills needed to complete the assignments set.

Early surveys, reviews of research on homework, and recent studies, all indicate common reasons for assigning homework. These include: repeating material that has already been learned in class (Brewster and Fager 2000; Cooper 2007); understanding if pupils have mastered the required skills (National Education Association 2008); introducing pupils to new material that will be presented in the next classes (Pytel 2007); and applying previous knowledge to new situations (Cooper et al. 2006). The amount of time that teachers spend on reviewing homework in mathematics lessons in middle and high schools in the United States of America (USA) has been found to be considerable; 15–20% according to two studies (Grouws et al. 2010; Otten et al. 2015). In terms of guidance for teachers on homework practice, Sinay and Nahornick (2016) stated in response that: “Overall, the focus of homework should be to improve conceptual understanding, fluency, and flexibility in math” (p. 20). Clearly, any design approach should aim to fulfil these aims and objectives.

Research on how homework is related to prior or subsequent teaching emphasises some key influences concerned with specific processes of teaching and learning. For example, the importance of homework is visible when teachers detect problems that pupils have during learning (Epstein and Van Voorhis 2001). Therefore, reviewing homework followed by knowledge and task correction at the beginning of a mathematics lesson may improve pupil achievement. This objective may be achieved in several ways: classes may begin with relatively long sessions of checking homework (as observed in mathematics classes in the United Kingdom and Germany); or begin with a quick review of a previous lesson (observed in mathematics classes in Japan). In all cases, however, some form of analysis of homework outcomes is present in almost all educational systems worldwide (Stigler and Hiebert 1999).

Based on a careful review of tasks, such educational activity may contribute to deciding how to adapt teaching materials in a lesson in order to overcome existing problems. Teachers in practice do not only use tasks and an analysis of pupils’ solutions in order to detect and locate faults, but also to understand and explore why they occurred, how to deal with them, and how to provide assistance to pupils in the following classes, so that they can address those problems.

In addition to the previously stated educational objectives and purposes of homework, it is very important to mention that goals and objectives are also conditioned by social and cultural contexts of the education environment within which teaching takes place. The Shanghai Nine-year Compulsory Education Curriculum Standards document (Shanghai Municipal Education Commission 1998) states, for example, that there are defining obligations related to homework; teachers should ensure that they carefully mark homework and provide feedback to pupils in a timely manner. This expectation can clearly be increased and further reinforced by school-level management, who can state that teachers are responsible for the collection and checking of pupils’ workbooks as part of their annual appraisal. Educational software design, therefore, should consider and potentially usefully accommodate such social and cultural concerns.

Parental concerns also influence practice in schools. In the eyes of some parents, a teacher of mathematics who fails to collect and mark pupil homework every day may not be a ‘good teacher’ and schools where this teacher works may then have a poor reputation. However, considering this need at an individual pupil level rather than a class level, as Deubel (2007) says:

“Realistically, differentiation of any kind is difficult to sustain when math class sizes tend to be large and when there are so many demands on teachers for preparing students for standardized testing. Implementing differentiated homework means getting to know your students better than you might now, and having them and parents understand a different view of “fairness”, particularly for grading purposes.” (p. n.p.)

Nevertheless, homework practice can be shaped by social expectations and the pursuit of ‘correct answers and scoring well’ in examinations, including being shaped in this way in the learning cultures of some countries (Biggs 1996). Such ‘correct answer and scoring well’ practices serve as an index, showing what is left to be learned, which is then subsequently used by teachers in their classes (Stevenson and Stigler 1992).

Previous research studies have shown positive results arising when pupils and teachers use web-based homework systems, and that their potential is not only to support the present curriculum but also to enhance the experience and understanding of subject topics (Bonham et al. n.d.; Loveless 1995). A range of studies have shown how certain software has been successfully integrated into different teaching practices to support homework activities. For example, the ASSISTment system, developed at the United States Department of Education and the National Science Foundation, gives teachers the ability to use detailed feedback and to tailor their instruction in order to focus on pupils’ needs (Razzaq et al. 2005). Similarly, Mastering Physics, a web-based physics homework tutor developed at the Massachusetts Institute of Technology, uses mastery learning to help pupils reach mastery levels when solving physics homework problems, as pupils can ask for hints on problems and receive feedback on common errors (Pritchard and Warnakulasooriya 2005). Additionally, pupils who use the Andes system, an intelligent tutoring system that provides support for problem solving for physics homework, can complete whole derivations, step-by-step, and receive feedback after each step (VanLehn et al. 2005). Leatham et al. (2015) state from their research that the success of mathematics teaching depends on the teacher’s ability to use all possible insights about pupil thinking to adjust teaching to that thinking in order to fully develop mathematical concepts. Their conceptualisation of ‘Mathematically Significant Pedagogical Opportunities to Build on Student Thinking’ seeks to improve effectiveness by closely examining classroom discourse. Knowing about the previous knowledge of pupils is considered vitally important if new understanding of mathematical concepts and ways of thinking are to be built. However, the influence of new and existing knowledge is also found to be related in an iterative ‘back and forth’ way, rather than following a single direction, from new to existing knowledge alone (Hohensee 2016).

Advantages of homework-assistance systems have been identified in the research (Brewster and Fager 2000; Nguyen and Kulm 2005; VanLehn et al. 2005), arising from the provision of immediate feedback to pupils, and automatic grading and recording of grades for instructors. Although these web-based homework-assistance systems can provide benefits, they can have disadvantages as well. Many of these systems do not consider pupils’ completed or ongoing work, as they require pupils only to enter a single answer for each problem; the system is providing feedback that responds to an incorrect answer rather than giving feedback about the issue that the pupil has. Consequently, teachers are likely to be less able to work out exactly where pupils are having difficulties and barriers to their knowledge, without seeing their work. However, such systems may have a role in motivating pupils to take homework more seriously, because they know it will be graded and the grade will be recorded.

The theoretical framework for the educational software design approach of eZbirka and the subsequent study of outcomes from use by teachers and pupils is based on a number of key findings from previous studies. In terms of a theoretical framework for the educational software design, opportunities afforded by educational technologies in the context of connecting learning between home and school were considered fundamentally. Research studies exploring efficacy have highlighted the support provided from the creation of ‘seamless learning spaces’ that will allow instruction (in Goos’s terms, appropriate tasks and interactions) and support (appropriate subject use to match goals or teaching methods) between the home and the school (Chan et al. 2006). Researchers have agreed that technology itself cannot provide a solid connection between the environments in which teaching and learning is carried out, but the manner of such implementation and the role of all involved in the educational process are of great importance (Cagiltay 2007).

In terms of efficacy studies, numerous studies have been reviewed which have sought to understand approaches, scenarios and methods needed for educational technology to connect learning across different settings (Lai et al. 2013). One system, called ASSISTments, aimed at:

“(a) encouraging students to rework problems they initially got wrong (and to enter revised answers), (b) focusing attention on the homework problems that students did not answer correctly, (c) reviewing correct solution processes for the problems that students found difficult, and (d) discussing common wrong answers to address underlying misunderstandings.” (Roschelle et al. 2016, p. 3)

The conceptual basis of the ASSISTments system was that: “Formative assessment involves using data from students’ independent work to give them helpful feedback and guidance while enabling the teacher to use the data to adjust instruction to meet students’ learning needs” (p. 1). Radović and Passey’s (2016) earlier study of the eZbirka system conceptualised the basis of the study in a similar way, by highlighting the shift in learning processes when digital technologies were involved in enhancing learning progress across informal, non-formal and formal learning environments. However, they applied a different methodological approach – a comparative analysis at a strategic and pedagogical practice development level within two different education systems (one in the United Kingdom and the other in Serbia). The methodological approach focused on teacher and pupil responses rather than pre- and post-tests comparing experimental and control groups. Findings from the Radović and Passey study (2016) provided a theoretical framework for this present study:

To address the key problem identified initially (pupils copy homework tasks rather than doing them on their own and providing the teacher with a real view of their abilities and issues), and to take advantage of the benefits of web-based systems, the educational eZbirka web platform was developed on the conceptual and theoretical basis stated in the section above. It was managed by the Mathematical Society of Serbia, and the entire project was funded by the Ministry of Trade, Tourism and Telecommunications within an initiative focusing on information society development in the Republic of Serbia. The process of software development, design and creation of tasks involved researchers from the Faculty of Mathematics in the University of Belgrade. They created a platform that supported homework activities, providing 13,000 task assignments on the platform. These assignments were aligned to the national curriculum, to teaching practice needs and to pupil learning goals in primary schools in Serbia (for pupils 11 to 14 years of age). All tasks were divided by grade, subject area and knowledge topics in order to facilitate ease of search and usage (as shown in Fig. 1).

The aims of the platform eZbirka in terms of teaching process were three-fold. They were to: (1) help teachers to understand the problems that pupils had, related to the content and understanding of mathematical concepts, relationships and abstract connections; (2) strengthen the relationship between teachers and pupils and to broaden the connection between learning at school and at home; and (3) prevent copying of homework between peers by generating a unique homework set for each pupil. The design of the platform was intended to fit the needs of pupils and teachers by refining, replicating, evaluating, and scaling. Its functionality was created to enable and enhance motivation of pupils to work independently and work better in achieving the goals of quality teaching (as defined by teachers themselves).

At the Faculty of Mathematics in the University of Belgrade, the Faculty of Natural Sciences in the University of Novi Sad, the Faculty of Education in Sombor, and in a number of other regional centres for teacher development throughout Serbia, workshops and educational seminars were held for teachers. Their aim was to educate and train them in appropriate use of the platform in their teaching.

The platform eZbirka is a web-based system, assisting teachers by specifying and providing homework tasks to pupils (as shown in Fig. 2). The platform contains lessons and pre-prepared packages of homework tasks that correspond to the national plan and programme for mathematics (The Institute for the Advancement of Education 2014b), regulated by the Ministry of Education and Science of the Republic of Serbia and The Institute for the Advancement of Education (the curriculum is compulsory for all schools and teachers).

Each prepared set consists of six different homework tasks. Homework sets are designed to cover the material that pupils are taught in particular teaching lessons and to examine all levels of competence that can be gained. Organisation of the tasks into groups, and their combinations, allow homework assignments to be selected by the system, which can be different for each pupil, but having similar problem formulations. The tasks in a group examine the same competencies of a pupil’s knowledge. In this way, the dynamics and variability of tasks in the homework set is ensured.

Pupils can access the platform through all types of modern digital devices, using a web browser. Addressing prominent factors that may represent barriers for the integration of ICT into lessons (Fethi and Inan 2010), access for pupils and teachers is facilitated to the maximum, without losing the structure of the work that the platform provides.

Pupils type a number in the field on the home page for learners, which refers to the homework their teacher gave them in the previous class. Based on this number, the platform generates a homework set. When solving each task, the pupil needs to describe the way the problem is being solved, and to highlight parts of the problem-solving task where there are doubts. In cases when pupils do not know how to solve the problem, the pupil needs to focus on providing an explanation - to define as much as possible what it is that the pupil does not know and what constitutes the problem to solving the task. Therefore, a pupil is always asked to think about the problem and the steps in solving it, and not just how to provide the final answer. After filling in forms for answers, pupils’ work is sent to the teacher, and pupils are able to check their answers instantly. Feedback to pupils is adapted, according to the nature of the task, and the pupil can compare answers with correct answers.

The teacher can review every homework solution and can analyse the responses, comparing pupils’ answers and solutions to the task. There is no automatic grading of pupils’ work. Teachers can have sight of the entirety of pupils’ solutions and ways of thinking, not just access to the final results and statistics. In this way, the teacher can easily perceive problems that pupils encounter and can carefully plan what will be done in any regular, additional or supplementary teaching class.

In terms of identifying efficacy of a homework-assistance system, a study by Roschelle et al. (2016) identified the impact on the mathematical abilities of pupils 12 to 13 years of age using a comparative statistical pre- and post-test experimental and control design. They found that:

“Results showed that the intervention significantly increased student scores on an end-of-the-year standardized mathematics assessment as compared with a control group that continued with existing homework practices. Students with low prior mathematics achievement benefited most. The intervention has potential for wider adoption.” (Roschelle et al. 2016, p. 1)

To explore behavioural outcomes in the context of the study reported in this paper, mixed methods were used, gathering evidence from pupils and teachers following implementation of the software eZbirka in primary schools using questionnaires. These identified whether the system tackled the existing problems in teaching interactions, shifting learning from ‘producing the right answer’ to ‘describing issues to support future learning’. The 620 teachers (all but about 15 from Serbia) registered to use the system has grown steadily over the past years. The research, conducted when the user base was smaller, used an evaluative approach, identifying the effects and contributions of web-based home and school link practices. Two questionnaires were developed in order to collect data from pupils and teachers.

The questionnaire for pupils comprised of sixteen rating-style questions and one question where pupils could offer their opinion about working on the platform, commenting or offering any concerns. Questions were adapted from the Instruments for Assessing Progress in Educator and Technology Integration (Knezek et al. 2002) and from Nguyen and Kulm’s (2005) research. The rating question items used responses on a five-point scale, with answer choices ranging from “Strongly disagree (1)” to “Strongly agree (5)” (see Table 1).

The questionnaire for teachers consisted of three parts. The first section focused on teachers’ attitudes towards uses of technology in teaching. Teachers were asked to rate their knowledge and skills when using ICT in educational processes. The second section related to teachers’ views towards the use of the platform eZbirka and its usefulness. The third section consisted of groups of questions that assessed and measured factors that need to be fulfilled for appropriate uses of ICT in teaching, mostly adapted from Islam’s (2013) prior research, with minor changes in wording, appropriately targeted for eZbirka use, to identify challenges of the Serbian curriculum and educational policies. A reliability test was carried out to determine internal consistency of items in the questionnaire using Cronbach’s Alpha reliability test.

The pupil questionnaire element of the study gathered evidence from four different primary schools in Serbia. On the platform eZbirka, a public call for mathematics teachers who used the platform eZbirka and wished to participate in a study with their classes was published. Four teachers agreed to join the study with their schools and classes. Teachers were trained on how to follow the requirements of the research reporting process, in the context of their daily usage of eZbirka. In these four primary schools, there were 198 pupils from eight mathematics classes participating (with the numbers of classes participating varying across the schools). There were 94 fifth, 66 sixth, 21 seventh and 17 eighth grade pupils (from 11 to 14 years of age); 92 were female and 106 were male. Pupils completed their homework on the platform eZbirka over a period of two months.

In the second part of the research, on teachers’ attitudes, 80 teachers took part. Using the same public call, teachers were asked to complete a questionnaire, regardless of whether they wished to participate in the study with their classes or not. Responses were collected from schools across Serbia, from teachers using the eZbirka platform in their teaching practices. They anonymously filled out a questionnaire available via the same web address as the platform eZbirka.

After each class, teachers gave a set of homework tasks to pupils. The pupils completed the homework, and the teacher examined the results of their work using the eZbirka interface before the following class - so the teacher was ready to resolve potential problems and doubts that pupils might have faced. Teachers were required to adjust the content of the following class to meet the needs of pupils and their perceived homework problems (see Fig. 3, which summarises the process). Pupils had automatic feedback after the homework was finished and were able to check their answers. After that, they could do the homework again and re-send it to the teacher (if they made mistakes as a consequence of spontaneous errors, or if the feedback helped them to overcome the problem presented). With the random task assignment feature, pupils were allowed to practice as many times as they wished. Each time a pupil repeated the homework task, the homework item was slightly different.

The study was conducted over a period of two months. After the test period, pupils were asked to complete the questionnaire regarding their perceptions and evaluations of the web-based practice. The teachers completed an online survey after a similar period of time using eZbirka.

Results of the research showed that pupils did not have any major problems in using the platform. Some minor problems that arose concerned the more complicated input of mathematical symbols, an issue that was corrected during the process of developing the working platform. In the study, no significant statistical differences were found between attitudes of boys and girls. Pupils’ responses to the questionnaire are shown in Table 1.

In order to investigate the reliability of the answer scores, Cronbach’s Alpha reliability test was used (Cronbach and Shavelson 2004). A value of Alpha greater than .8 indicates good reliability, less than .6 is poor and less than .5 is unacceptable. The value of Cronbach’s alpha for this questionnaire was high, at .838. Table 2 shows the means and standard deviations of the responses, which will be discussed in more detail.

Results showed that pupils were provided with a new learning experience, i.e. using computers to solve mathematics problems and the opportunity to practice more to achieve better scores. The survey results indicated that pupils felt that they could successfully work with a computer (M = 4.60, SD = .612), but that they were not so confident when it came to using computers for learning mathematics (M = 3.56, SD = 1.096). Before using eZbirka, pupils had previously used a computer to make presentations or to explore mathematical objects, or to create posters for schools (although these activities were very rare, according to teachers, happening perhaps once a year). Based on the comments selected from their open responses, it can be concluded that they enjoyed using eZbirka:

6 ^th grade pupil: “Finally, we use a computer to learn mathematics.”

8 ^th grade pupil: “eZbirka is a good way to use the computer. Using it, we feel as we have additional lessons.”

7 ^th grade pupil: “eZbirka helped us to improve marks. You [pupil speaks to the teacher] should earlier to tell us for this, we would have probably better grades from the beginning of the school year.”

The pupils pointed out that the homework provided from eZbirka was more interesting than ordinary tasks and they positively highlighted the clarity of the requirements expected of them (M = 4.51, SD = .811; M = 4.20, SD = .890). The results showed that pupils identified immediate feedback as being valuable and agreed with the statement that feedback helped them to recognise their mistakes. Afterwards they said they had insight and confirmation of their understanding and they knew their performance after each class. The findings showed that pupils often after feedback presented the homework to the teacher - stating that feedback encouraged them to examine their own mistakes and make corrections in solving their tasks. A significant majority of pupils believed that the homework in eZbirka helped them to learn more (M = 4.54, SD = .744):

6 ^th grade pupil: “I really like working on eZbirka. It is easier to learn; we can do more tasks and immediately see whether we are doing well. With the help of computers we can do better than to practice in the classroom.”

5 ^th grade pupil: “eZbirka helped me to understand math better and be more active - I feel free to ask questions now.”

The findings showed that pupils were more motivated to work and learn mathematics (M = 4.32, SD = .938), stating that they preferred to do their homework using eZbirka, and were more confident in their work. The platform eZbirka encouraged them to practice more; pupils with limited understanding in some topics were able to receive feedback, which enabled support in overcoming barriers in their learning. Feedback also supported pupils who wanted to increase their performance, giving them an opportunity to reach maximum scores. Finally, pupils were confident that eZbirka helped and supported teachers in gaining understanding of their mathematical concepts and misunderstandings (M = 4.51, SD = .759).

8th grade pupil: “I have already got used to this kind of work.”

Results of the first part of the teacher questionnaire showed that teachers had a positive attitude to the use of information technology, as they saw it as a means of socialisation and networking that was necessary and desirable for uses in classes (n = 75; 93.8%). All teachers agreed that the use of modern technology in teaching was necessary. Although teachers felt that their competence and abilities to use modern technology in teaching were high and satisfactory (n = 71; 88.7%), they also felt that they needed additional training in order to adequately and appropriately use computers when teaching mathematics and in order to acquire self-esteem and greater degrees of skills (n = 48; 60%). In an open question (how and in which way they used information technology to teach), the most common responses were:

“I apply my competence by making digital materials (PowerPoint presentations and Geogebra applets) that I need for classes. I maintain my own blog, as a tool of communication, primarily with my pupils, but also as a ‘warehouse’ of materials needed for classes.”

“I try to find and use a variety of interactive features for teaching, which enrich the educational process. Not only in terms of production for a given presentation on the theme, but also to connect via social networking site and to set the problems and tasks. As a math teacher, I try to introduce math to pupils as an interesting subject using all available content that can be applied in the classroom.”

“Preparation of Power Point presentation for teaching, the use of pictorial, audio-visual materials which can be found on the network, using the existing program for creating quizzes, crosswords and associations.”

Prior to use of the eZbirka system, most of the teachers created digital materials by themselves or re-shaped already-existing ones, shared them with colleagues, and used materials that were available on the Internet (due to the shortage of adequate high-quality electronic interactive teaching materials). Most respondents stated that the great difficulty in applying ICT was the lack of sufficient interactive material, especially when it came to the subject of mathematics (n = 60; 75%). The results of the second part of the questionnaire indicated that they felt that eZbirka was addressing this deficiency. Respondents believed that the use of the eZbirka platform encouraged pupils, built relationships between pupils and teachers, and helped a pupil be more engaged (n = 77; 96.3%).

Teachers pointed out that the important functionalities of the homework facilities were to ensure the variability of tests and automatic feedback to facilitate the process of problem solving. As well, teachers believed that pupils were motivated to work (n = 76; 95.0%) because they were sure that they could not then copy the homework and that their work could be evaluated by the teacher. The questionnaire results indicated that teachers were of the opinion that eZbirka helped them to reveal gaps in knowledge of pupils and to successfully plan the next curricular activities (n = 78; 97.5%). Teachers emphasised that the need for pupils to comment helped pupils to think about their knowledge, and to formulate ways to explain how they solved the task. Therefore, they felt pupils were open and willing to express their opinion in regular classes, and they felt more secure about their knowledge. Teachers said that pupils’ comments on tasks provided them with important indicators in identifying gaps in the pupils’ knowledge at an early stage of learning. According to the observations of teachers after a short period of usage, within eight weeks, pupils could build a more positive attitude towards work and teaching activities.

The results of the third part of the teacher questionnaire considered how to improve and facilitate the use of ICT. The teachers’ assessments of the motives for the uses of ICT confirmed the responses from the first two parts of the study and confirmed the views and key points of findings presented in this paper. The question items were recorded using a five-point scale, with answer choices ranging from “Strongly disagree (1)” to “Strongly agree (5)”. Respondents expressed strong positive outcomes to the questionnaire statements; all measures were ranked very highly. Table 3 presents the means and standard deviations of the responses of teachers.

The results of this study indicated that these teachers, in terms of the most important motives for the use of ICT in teaching, were concerned with: raising the quality of teaching and encouraging motivation of pupils; and supporting greater engagement of pupils. Furthermore, the results revealed that respondents were willing to use different and modern learning environments in non-traditional ways, to integrate new technology with their pedagogy, into their curriculum and for learners. The teachers were aware that the aim of reaching these goals required twenty-first-century competences to be developed, and they were willing to continue to develop these through personal education and professional training.

Another important fact that testifies to teachers’ attitudes is that, after using the platform eZbirka, all of these teachers continued to use eZbirka beyond the test period. Moreover, teachers encouraged other mathematics teachers from their schools and districts to include use of the platform eZbirka in their daily teaching activities.

The research questions initially asked: whether the educational software developed addressed the problem identified, according to teachers and pupils; if so, how it did this, and to what extent; and how effective teachers and pupils rated the system, and why. It is clear from the results shown above that the educational software did address the problem of pupils not completing homework tasks on their own, and not enabling teachers to understand their needs with tackling future mathematical tasks.

The Serbian National Plan for Educational Use of Information and Communications Technology (National Education Council 2013) lays out strategic policies and proposals in order to promote meaningful and collaborative learning and to develop learning-to-learn and other competencies required in the twenty-first century. The challenge has been to design and implement a system that successfully bridges the opportunities between home and school, regarding development of understanding and effectiveness of learning, making improvements on former behaviours that have demonstrated particular disadvantages. Much previous research has shown limited success of homework systems in helping formal education, suggesting that a technology used needs to be part of a broader infrastructure of information and support (Cooper 2007; Kerawalla and Crook 2002; Lewin et al. 2003). The Roschelle et al. (2016) study demonstrated the attainment benefits of such a system, while this study has demonstrated the pedagogical, social and learning behaviour benefits that have arisen. The developers of the eZbirka system have created a system that fits into and integrates with the structure of teaching, respecting didactic principles and relying on regulations and laws required by the relevant ministries.

This research has examined: (1) the attitudes and outcomes of pupils using computer technology connections between home and school learning activities; and (2) school teachers’ usage and outcomes of the computer technology system eZbirka. The study showed that teachers did not use the task analysis and task solutions just for detecting and locating faults, but to understand and explore the reasons for their pupils’ problems; they learned how to deal with pupils’ problems, to provide assistance to pupils in subsequent classes so that they could address misunderstandings. eZbirka was valued as a tool that solved the behaviour of cheating and homework copying.

The results of this study indicate that pupils are managing well in working with computers, but do not yet use them fully for learning purposes. However, the study revealed that the innovative practice of assigning homework and work through the platform eZbirka is not difficult for pupils to manage – moreover, pupils say it makes learning more interesting. Pupils positively reported on all aspects of the platform: displaying correct answers; the possibility of resubmitting it to the teacher; commenting on the tasks; and offering questions for the teacher. Pupils can now practice more than they did before. The positive attitudes and the willingness of pupils to be engaged in e-learning homework activities suggest that future e-learning initiatives designed carefully through an understanding of the conceptual and theoretical bases of needs, have great potential. Furthermore, teachers reported significant positive attitudes toward the use of information technology. The findings indicated that teachers were aware of their limited knowledge regarding the use of computers in teaching. However, teachers were of the opinion that eZbirka as a form of information technology could provide a significantly important level of adaptation to the teaching process.

The need for provision of appropriate training to make ICT-supported learning more widely accepted in educational processes, involving the development of ICT expertise and content for practical use in teaching, needs more exploration. This study examined factors that can influence teachers’ levels of ICT usage. Teachers reported highly and positively on all factors necessary to increase and ensure the use of computers in teaching: providing better equipment; providing technical support to the school; providing more interactive teaching materials; intensively promoting online resources and materials; and increasing support of the school administration. Some technological issues were identified, however, and these clearly needed to be resolved.

It is certain that any change in the educational environment must be undertaken carefully, to promote effective teaching and learning processes, both at individual and at global levels. ICT in education is not about the specific kind of technology available to teachers or learners, it is about having the effective kind of equipment that can improve teaching methods and increase confidence in knowledge development. As the National Council of Teachers of Mathematics (NCTM 2017) have stated: “Effective teachers optimize the potential of technology to develop students’ understanding, stimulate their interest, and increase their proficiency in mathematics”. The findings of this study reveal that the goals and purposes of the educational software eZbirka were achieved in this way, and that its adaptation within education can add to purposeful teaching. These are important conclusions, and its application to other contexts where similar problems exist may well enable pupils and teachers (and parents) to benefit.