Does lego training stimulate pupils’ ability to solve logical problems?

Abstract

The purpose of this study is to investigate the effect of a one-year regular robotic toys (lego) training on school pupils’ performance. The underlying pedagogical perspective is the constructionist theory, where the main idea is that knowledge is constructed in the mind of the pupil by active learning.

The investigation has been made in two steps. The first step was before the treatment and the second after treatment. For both cases we have constructed and included control groups. The data was gathered from different pupils from two different age categories, from different classes, from different schools, and finally from different places in Sweden. We have investigated whether the approach of involving the lego training in the schools activities might lead to improving the adoption process and that the pupils would perform better in mathematics and technique. Our null hypothesis states that the lego robots do not have a positive or negative effect on the pupils’ ability to solve mathematical and logical problems. A one-way ANOVA test leads to acceptance of the null hypothesis. However, when ANOVA test was performed on sub groups of pupils, the null hypothesis was rejected in some cases. This indicates that lego training may be useful for some groups of students. Furthermore, a hypothesis test regarding certain correlation measures was conducted, supporting this theory. In general, the statistical analysis suggest that there is no obvious over-all effect of lego, though there are significant positive effects of lego for sub groups of pupils. In all, we find the results promising enough to suggest a larger experiment to be performed.

The pupils have different learning styles in their approach to LEGO training. The role of the teacher, as a mediator of knowledge and skills, was crucial for coping with problems related to this kind of technology. The teacher must be able to support the pupils and to make them understand the LEGO Dacta material on a deeper level.

Introduction

This research project called “Programmable construction material in the teaching situation”, aims at studying the pedagogical effects caused by the application of LEGO Dacta materials in some schools in the area of middle Sweden. For that purpose, we took two groups of study in order to make comparisons. One of them is called the control group (CG) formed by schools that have not been subjected to the experimental work with LEGO Dacta and the other called experimental group (EG) that was represented by schools that have participated pedagogically with the LEGO Dacta proposals and technological materials. LEGO Educational Division, part of the LEGO Group, has been developing and producing educational materials for schools for a number of years. ROBOLAB/LEGO Mindstorms for Schools – hereafter referred to as LEGO Mindstorms for Schools – combines the very latest robotics and measurement technologies with traditional mechanics.

The starting point is a former project called INFOESCUELA, which was a pilot project that began in Peru 1996 (Iturrizaga, 2000). It was initiated by the Peruvian Ministry of Education, with the objective of introducing technology to primary schools through the use of Lego Dacta materials. Over the course of three years, from 1996 to 1998, the project was expanded to cover 130 schools throughout the country of Peru. In general terms, the result of the Peruvian project showed empirical evidences that the pupils from the experimental group successfully fulfilled their school activities due to the use of LEGO Dacta material. In the referred case, the EG gained better percentages of achievement than the CG in the tests of mathematics, technology, Spanish and eye to hand coordination. The results of this research were so encouraging that LEGO Dacta was interested in studying the effects of using their products in Sweden, which represented a different school context than in Peru.

We were conscious of many of the dissimilarities between the school contexts of Peru and Sweden, and especially the more complex evaluation process in the Swedish school environment. In Peru, the chances were good to find control groups of pupils that never have been in touch with computers. Due to an over all higher technological level in Sweden, we could not expect to find pupils that were totally inexperienced with computers, because most children come into contact with computers at early ages in their families and elsewhere in the community. The Peruvian research project was in that respect of a more simple nature, and the results could not easily be applied to other contexts, i. e. the circumstances of the Swedish case were more subtle.

Gustafsson and Lindh (2001) found that most pupils thought it was fun to work with programmable lego teaching materials. Their study was included in the “Construction kits made of Atoms and Bits” (CAB) project and involved pupil in grades one to three working with programmable lego tools. However, the question is whether LEGO Mindstorms for Schools is a useful tool in the area of technology that is also suitable for slightly older girls and boys in compulsory school. This study concerns pupils in the age of 11–12 and 15–16 years. And the focus is not on attitudes, rather the study is about problem solving. Our research question is: Might lego training lead to improving the adoption process and might pupils perform better in the schools especially in mathematics and technical tasks?

The main ideas underpinning the project adhere to constructionist theory, according to which knowledge is not just simply transmitted from teacher to pupil, but rather constructed in the mind of the pupil in the form of active learning (Gustafsson and Lindh, 2001, Harel and Papert, 1991). According to constructionist theory, the pupil is particularly likely to create new ideas when actively engaged in producing external artifacts which can be reflected upon and shared with others. Papert (1980), a pioneer in the area of programmable educational material for children, is an advocate of constructivism in education; i.e. constructionism. The concrete material which the pupils work with is robotic toys that can be programmed by computers (RoboLab 2.01).

LEGO and LOGO rely on the same pedagogical principles, even if LEGO can be considered as an extension of LOGO. Papert wanted to design a suitable computer language for children, which had the power of professional programming languages, but was easy enough for non-mathematical beginners. The name LOGO was chosen for the new language to suggest the fact that it is primarily symbolic and only secondarily quantitative (Papert, 1980). LOGO has been developed over a period of two decades. It is an extraordinary programming language in the sense of flexibility that gives special characteristics for discovery learning. In short, LOGO, is based upon two ideas; the first one is that students can learn more effectively through experiencing and discovering things for themselves, the other assumption is that the computer is a perfect medium for discovery learning.

Due to their programmability and transparency, robots and programmable bricks are among digital toys that today offer specially interesting features. These toys, in advanced forms, can be given certain characteristics, like agency and identity, that make them behave like living entities, challenging our way of thinking to life, as they position themselves on the boundary of what is animate and inanimate (Turkle, 1995). Their hybrid nature makes it possible to play out the fine line between objectifying minds and animating things (Ackermann, 2000).

Cybernetic construction kits, integrating the physical building of artefacts with their programming, can encourage the development of new ways of thinking (Resnick, 1996a, Resnick, 1996b) that feeds new thoughts on the relationship between life and technology, between science and its experimental toolset (Resnick, Berg, & Eisenberg, 2000), between robot design and values and identity (Bers & Urrea, 2000). As constructionism supporters argue, thanks to these objects many concepts that are usually considered a privilege of adults, who can deal with symbolic and abstract knowledge, are made accessible and comprehensible for children as well (Resnick, 1998, Resnick et al., 1998).

During the same period, Marvin Minsky was encouraging children in the new MIT Children’s Laboratory to learn to control a small robot which moved on the floor and could draw a record of its movement with a pen. It seemed to Papert and Minsky a logical step to integrate control of this robotic turtle into LOGO, which later was replaced by a screen turtle, but the language remained true to Papert’s ideals; that it should be a tool for learning concepts like planning, problem solving, and experimentation.

Mitchel Resnick, professor at the lab and director of the Lifelong Kindergarten research group, compared and contrasted several projects chosen by workshop participants which featured the construction of physical objects, often build from LEGO bricks, that were controlled by the computer (Resnick, 1991). For Resnick each project was rich in personal meaning, as it symbolized something for each of the project leaders, and secondly there was a great diversity among these projects. Diversity of themes showed itself through the variety of projects, none of them “typical”. He concludes that while both LEGO and LOGO encourage open-ended construction, their use alone does not lead to diversity. Participants were encouraged to forget constraints when choosing a project theme.

Many projects were performed, but most of them were small, in terms of size of sample of individuals and length of study. Criticism has been raised towards these kinds of studies, as being only stories rather than research. The same stands for most of the reported LEGO projects. Most of them are small case studies which have been run over a short period of time, some weeks or up to a couple of months. For example Erwin, Cyr, Osborne, and Rogers (1998) report an interesting study about how to create educational tools that allow K-12 students to engage in various engineering projects. Another study, Järvinen (1998), reports interesting, though rather vague, results on using the LEGO/LOGO learning environment in technology education.

Becker (1987) emphasizes the importance of testable consequences in research about LOGO, in his discussions of advantages and disadvantages of two types of research methodologies used to study the effect of LOGO in classroom settings: the treatment methodology and computer criticism. Several researchers, including Becker (1987), have stressed that the main evidence showing that LOGO can produce measurable learning when used in “discovery” classes has been obtained in situations close to individual tutoring. In normal-sized classes, the evidence clearly shows the need for direct instruction in the concepts and skills to be learned from LOGO, as well as further direct instruction to enable students to generalize what they have learned for transfer to other situations. This is in complete opposition to Papert’s conception of the discovery approach to LOGO. Pea (1985) asks: “What contribution do we get from the computer to our mental functioning?” Whether the computer is only an amplifier of cognition or a reorganizer of our minds.

Moore and Ray (1999) argue for more advanced formal statistical methods for sensitivity and performance analysis in computer experiments. They suggest, for example, multivariate methods in analysis of thorough investigations. Our research project aims to combine different kinds of methods in a more broadly one year study.

To get a fair comparison we wanted both groups, EG and CG, to be equivalent regarding educational, social and demographic characteristics. Different educational tests were applied to these groups. When recruiting “experimental” classes (EG) our ambition was to reach a balanced distribution with schools in varying geographical regions in the middle of Sweden and representation of schools in small, medium sized and big municipalities. Our intention was also to get an equal proportion of classes in fourth grade and eighth grade. The class sizes varied between 17 and 43 pupils.

The project took part in 12 “experimental classes” of different schools in the middle of Sweden. Together there are 322 pupils: 193 pupils in the fifth grade (12–13 years old) and 129 pupils in the ninth grade (15–16 years old). There were 12 “control classes”. All together are 374 pupils, distributed on 169 pupils in the fifth grade and 205 in the ninth grade. The training time was around 2 h a week during 12 months, from April/2002 to April/2003. So the exposition time is around one year.

The LEGO material, i.e. the construction kit, consisted of a mechanical assembly system, a set of sensors and actuators, a central control unit (the programmable brick), a programming environment, i.e. a computer and software and working instructions and manuals. The programmable brick is the most noticeable component of the kit: it provides both control and power to all LEGO constructions. When the experimental classes worked with the LEGO material, pupils cooperated in smaller groups, generally 3–4 pupils, which we called working groups. The groups did not follow a certain syllabus working with the LEGO material. The work was rather adjusted to the ordinary school activities and to the local preconditions at each school. Just to be sure that the classes started in a similar way, the teachers were instructed to start up with the same kind of task.

It was stated in the research plan that the pupils should work with the LEGO material about eight hours a month. This level was decided to make sure that the pupils should have similar preconditions. It was planned that the researchers should come out and visit the EG during this period, to follow up how well the work continued and what eventually was changed during this time. We have experienced that the school classes have solved the practical things in different ways. The physical preconditions concerning rooms have varied at the different schools. Some classes could have LEGO material permanent, while other classes pick out the material each time before using it. Some had to work in special computer rooms, other classes needed to go to other school rooms to get computers, and other classes had only a couple of computers in their ordinary class room which they used to program the robotics the pupils built.

In some classes, especially in grade 8, the teachers have been able to integrate the work in their ordinary teaching, in mathematics or in technology. This was not usually the case in grade 4, where LEGO work was more like a separate part of the school day. The participating teachers in the project have been taught how to handle the LEGO Dacta material, to prepare them for their pupils often highly intricate questions.

In this research project we used both qualitative and quantitative research methods.

The qualitative methods used in the project were observation, interview and inquiry. Every teacher involved in the project had to document their LEGO lessons; what kind of activities were done during the lessons, how the pupils worked, what kind of problems they faced, and so on. The most important task for both teachers and researchers involved in this project has been to document and report the research experiments and findings as thoroughly as possible by providing detailed descriptions of cognitive processes and concepts developed by the children. In this way, the collected information should make it possible to understand how children act, think and accept new challenges, and how they seek and find solutions to problems that arise. The purpose of data collection has been to successively create a picture of the way children develop their knowledge. The role of the teacher, as a mediator of knowledge and skills, was crucial for coping with the shortcomings of the available technology. The teachers also engaged in the documentation of children’s activities as an integral part of their everyday work. This documentation was produced in a variety of formats, like texts, images, video, etc.

By means of the reflections and interpretations adults made on the issues that emerged from the work with the children, it was possible to take into consideration the “theories”, although preliminary, that the children had developed during their experiments with LEGO MindStorms and with the subsequent versions of the construction kit. Our aim was to collect descriptive and narrative elements on what happened in the encounters between the children and the construction kit, and how the situation developed. Therefore, the documentation was viewed as a process of reflection and elaboration, enabling us to obtain from the children the requirements for the evolution of the LEGO kit.

The submitted documentation were filed and stored for subsequent analyses for the research findings. The researchers made regular visits to the project schools. While visiting they made interviews with the pupils and observations of the work. The interviews were taped for later analysis. The purpose was to be able to follow the working groups achievements during the project as well as discern their attitudes/feelings about the LEGO material. Also we could analyse their way of speaking about the material and their understanding of the LEGO concepts.

The quantitative methods consisted of different tests in mathematics and problem solving, and also covering inquiries concerning pupils’ attitudes to working with robotics.

Before the school classes started their work with the LEGO Dacta material they had to perform a test in mathematics that was similar to the Swedish national test in mathematics, followed up by a test at the end of the project. The test was done by both EG and CG. The purpose was to compare the achievements for the two groups and see if there had been any changes/improvements. The problem solving test for both EG and CG at the same period was given for the same reason.

The relevant question here is whether the lego robots have stimulated the pupils ability to solve logical problems or not. In other words, our null hypothesis states that the lego robots do not have a positive or negative effect on the student’s ability to solve mathematical and logical problems. Formally, we may write the pupils test scores as a linear function as follows:

$$y_{\mathit{ij}}={\mu }_i+{\epsilon }_{\mathit{ij}}\text{,}\left\{\begin{array}{c}i=1\text{,}2\hfill \\ j=1\text{,}\dots \text{,}n\hfill \end{array}\right.$$

where y_ij is the test score for the (ij)th student, i = 1 if the student has been working with lego robots, 0 else, and n is the number of available test score results. ε_ij is a normally distributed stochastic error component such that $V[{\epsilon }_{\mathit{ij}}]=V[{\epsilon }_{{\mathit{ij}}^{\prime }}]={\sigma }_i^2$ for j ≠ j′. The population mean may be written μ_i = μ + τ_i where τ_i is the specific mean, or equivalently, the factor, representing the lego effect. The null and alternative hypotheses may be written

$$\begin{array}{cc}\hfill & H_0:{\tau }_1={\tau }_2\hfill \\ \hfill & H_A:{\tau }_1\ne {\tau }_2\hfill \end{array}$$