In the last decade, Autism Spectrum Disorder (ASD) prevalence rate has significantly increased, which consequently led to the expansion of research and expenditure in the field, predominantly focusing on searching for the cause. In a typical classroom scenario, working with children with ASD very often requires 1:1 teacher to child ratio, which makes it very expensive and difficult to implement. Serious games have been utilised as a medium for teaching various developmental skills, such as social interaction, speech, motor skills development, emotion recognition, and other basic concepts. Designing serious games for ASD population differs from other games and even other serious games significantly. It requires a holistic approach with extensive knowledge and expertise from fields other than computer science, such as psychology, sociology and cognitive science. However, once harnessed correctly, such games can be used by children with ASD on their own time, with or without supervision and they can be educational. In addition, they can adjust the appropriate pace while at the same time providing feedback in form of reinforcement and correction. Applying the rules of science of learning and teaching, one can design games that are educational for all types of learners, including children with ASD. In this paper, two independent user studies have been conducted, demonstrating how serious gaming and e-learning principles can be harnessed in order to intervene, develop or strengthen pivotal developmental skills, like learning novel vocabulary, counting, identifying numbers and colours, and responding to inference questions. We have tested the educational e-book with children diagnosed with ASD and with typically developing children to assess skill acquisition in native language for children with ASD and in English, a foreign language, for typically developing children to demonstrate the educational aspect of the game for all types of learners. We showed that the same e-book in two languages can be used for teaching different types of learners through a fun and engaging medium.
Introduction
Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that affects three core areas of development: language and communication, social skills, and ones interests/behaviours. It is a disorder that begins early in development, before age 2 (Luyster et al. 2005), characterised by the lack of language and social skills development and development of patterns of rigid, repetitive, auto-stimulating behaviours that interfere with overall functioning of a person, Association (2013). Children with ASD continuously have demonstrated major impairments in these key building-block skills (Smith and Bryson 1994; Dawson et al. 2004; Katarzyna et al. 2010), and the absence of the same has often been used as an early predictor of ASD (Rozga et al. 2011). In addition, they often exhibit a difficulty in processing sensory information, by under- or over-processing different inputs which, as a consequence, affects their learning. This in turn leads to difficulties in developing more complex language and social skills, required for successful inclusion into society.
Serious gaming, a process of playing video games for purposes other than entertainment or leisure, has contributed to several fields, such as education (De Freitas 2006), simulation (Shelton and Hedley 2002), health (Arnab 2012) and others. The advantage of using serious games in training and education is that it exposes users to deeply engaging, visually dynamic, rapidly paced, and highly satisfying experiences, in comparison to conventional teaching and training methods. At the same time, studies have demonstrated that children with ASD acquire skills faster when instruction is delivered via computer vs. the traditional teaching strategies, and are more interested and motivated to learn through computer-assisted instruction (Whalen et al. 2009; Chen and Bernard-Opitz 1993). Consequently, several studies in serious games community have contributed in this domain (Zakari et al. 2014) showing a significant potential of this medium on detection and intervention for ASD. While none of this work, to the best of our knowledge, is based on the three-term contingency—the learn unit (Skinner 1958; Greer and McDonough 1999; Greer 2002), a predictor of all learning, we tried to design and develop our work based on these findings by using them as a theoretical foundation for the game design.
Anzeige
In this paper, the development process of the interactive educational e-book is presented, along with the two user studies investigating how the e-book can be utilised for teaching children with and without ASD various developmentally appropriate skills. The first study investigates how children with ASD acquire basic learning concepts (i.e. pointing to colours and numbers or basic vocabulary from the story) in their native language, by going through the e-book and whether they are able to transfer that knowledge to other environments, making it functional knowledge. Children with ASD have limited vocabularies and deficits in language and communication, so this is why we wanted to test whether they can acquire language and cognitive concepts through game playing. The second study explores whether the same e-book can be used for teaching typically developing children same concept but as a foreign language vocabulary. These children in comparison to children with ASD are fluent speakers in their native language and have rich functional vocabularies so we decided to test acquisition of novel language skills. So, the main goal was to test whether the e-book and the game can be utilised as an educational tool for children with and without disabilities.
The main contribution of this work is showing that serious games developed on the findings from behavioural science can be very effective in teaching children with and without developmental disorders. The results from both studies show that the acquired skills can be efficiently transferred to new media and environments. In addition, this work successfully bridges the gap between the two disciplines and promotes incorporating technology as part of the educational and pedagogical process in all schools. The studies on the process of the skills acquisition, done with children one on one, require extensive time because experimenters collect all data, in addition to the computer keeping the score. Furthermore, the generalisation probes have to be done with children in natural environments through play, so the sample sizes in such studies cannot be large due to such intensity and tracking every learn unit/interaction.
Related work
Technology within the research on ASD has been used for detection and measurement of psychophysical, physiological and neurological states. There is evidence that electroencephalography (EEG), skin conductance / electrodermal activity (EDA) and heart rate, which can be measured with photoplethysmogram (PPG), can be used for autism detection and monitoring of the emotional state (Filipek et al. 1999; Coben et al. 2010; Critchley 2002; Goodwin et al. 2006). Research in physiological feedback has shown that discomfort caused by looking at faces and making eye contact in population with ASD can be detected through EDA measurement and hyperarousal of associated brain regions (Joseph et al. 2008). Similarly, heart rate has been used for measurement of the arousal modulation in individuals with ASD (goodwin2006cardiovascular). Picard et al. (2001) combined four different simultaneous measurements skin conductance, heart rate, electromyography and respiration, and extracted eight different emotions using pattern analysis, with the 81% accuracy.
Other studies investigated the effect of technology and computers on learning and student motivation. It has been shown that children with autism can master more educational objectives using a computer, compared to traditional teaching strategies, and are more interested and motivated to learn through computer-assisted instruction (Chen and Bernard-Opitz 1993; Whalen et al. 2009). Price investigated the effect of the delivery medium on children with autism (Price 2011). He compared comprehension and information acquisition when reading interactive e-books (on iPads) compared to printed books and found positive outcomes for each participant using e-books. The same study revealed that the reading on iPads was motivating and reinforcing, which consequently decreased students’ inappropriate behaviour in the classroom.
Anzeige
Bosseler and Massaro (2003) developed and tested via two separate experiments a computer-animated program for teaching children with ASD vocabulary and grammar. They found that children with ASD learned novel vocabulary with a computer-animated agent and they noted the active participation of children in the process of learning, transfer and use of the novel language in a natural, untrained environment, which is the key of learning any skill.
Kebritchi and Hirumi (2008) in a comprehensive educational games review pointed out that games of this type are designed taking into account several learning theories, such as learning by doing, discovery learning or constructivist learning. Skinner (1958), looking from a point of view of the science of learning, pointed out how in video games player’s behaviour is guaranteed to be reinforced, because the player contact salient and immediate consequences, and are almost guaranteed to be successful. The importance of the immediate feedback a teaching machine can provide vs. teacher teaching a large group of students and never being able to do so, was emphasised by Pressey (1926) in the 1920s as a main contribution of computerised instruction to the education. In addition, enabling students to advance through curricula at their own pace, permitting them to play an active role in learning vs. passive listener role is shown to be highly motivating for a learner (Pressey 1926; Skinner 1958). The feedback, being a correction and leading or prompting a player to a correct response or being a positive reinforcement to strengthen the responses in the future, seems to be a key to learning (Mory 2004; Skinner 1958; Albers and Greer 1991; Greer 2002). The consequence, a component of the three-term contingency or a learn unit is what effects the behaviour in the future. The appropriate and immediate consequence based upon student behaviour, when properly delivered will increase the likelihood of future correct responding to the same or similar stimuli. Therefore, the intact learn units (LUs), complete antecedent–behaviour–consequences, should be the aim of all instruction, computer assisted or not. The interlocking three-term-contingency has been a proven measure of a teacher (i.e. machine)–student interaction upon which all learning is based in Greer and McDonough (1999), Albers and Greer (1991) and Greer (2002).
There are many examples of serious games that have been proven to be efficient in achieving their goals while providing a fun, engaging and informal, game-like environment. Game-based learning and educational serious games combine gaming and learning, and are proven to be effective way of computer-aided learning (Johnson and Jonassen 1988; Squire 2003). A study by Kebritchi et al. (2010) showed that modern mathematics computer games have positive effects on mathematics achievement and class motivation. In another example, the Re-Mission serious game allows young people with cancer to learn about it and its treatment (Beale et al. 2007). A systematic and extensive review of serious games, with an emphasis on the positive aspects of gaming on learning and skill enhancement, has been presented in work by Connolly et al. (2012). In their paper, they showed that playing computer (serious) games can have multiple positive cognitive, behavioural and motivational effects. Another study by Annetta et al. (2009) showed that such games can enhance engagement and motivation in children with learning difficulties or attention disorders.
Another, closely related, subcategory of serious games are those mainly used for health treatment. These serious games are not used only for treatment of diseases, but also for physical, mental, and social well-being treatments. These include obesity (Scarle et al. 2011; Navarro et al. 2017), phobias (Botella et al. 2011; Kwon et al. 2013), diabetes (Fuchslocher et al. 2011; Nauta and Spil 2011), autism (Hourcade et al. 2012; de Urturi et al. 2011; Hulusic and Pistoljevic 2012) and many others. A complete literature survey on serious games for health can be found in work by Wattanasoontorn et al. (2013), that also classifies them by game purpose, functionality, stage of disease and by player’s wellness and analyses each game based on different functionalities. Serious games for ASD treatment are mainly aimed at teaching basic developmental skills, including communication, social behaviour and conversation, imaginative skills, sensory integration, emotional expressions, etc. (Hourcade et al. 2012; de Urturi et al. 2011; Hulusic and Pistoljevic 2012, 2013, 2016; Pistoljevic and Hulusic 2017). An extensive overview of serious games for treatment of ASD, along with a classification according to technology platform, computer graphics (2D vs. 3D), gaming aspect and interaction type, is presented by Zakari et al. (2014).
Finally, Smeets and Bus (2012) examined whether animated e-books, where motion pictures, sounds, background music and interactive labels are used instead of static images and text, aid word learning and story comprehension by comparing outcomes for kindergarteners across four experimental groups: static e-books, animated e-books, interactive animated e-books, and a control group that just played non-literacy computer games. They found that children acquired most vocabulary after reading interactive animated e-books, followed by non-interactive animated e-books and then static e-books.
Before the e-book
This work presents the latest version of the “Wolfie Wolf” series. There were several predecessors of the interactive e-book presented in this paper. The first incarnation of the story was in a form of a theatre play, initiated by EDUS—Education for All organisation and directed by Belma Lizde Kurt. The screenplay and the children’s book storyline was written by Fekete–Sullivan (2015). The play was then created in co-operation with the Aparat Theatre and SARTR (Sarajevo War Theatre), and has been performed several times by children actors with and without developmental disorders, Fig. 1.
Fig. 1
A scene from the theatre play
×
The second appearance of the Wolfie was in a hard copy children book with tactile stimulations and interactive elements (Fig. 2). The book was designed to provide multisensory stimulation for 3–12-year-old children with and without disabilities and to reinforce comprehensive development and school readiness skills. The book was bilingual, designed in Bosnian–Croatian–Serbian (BHS) and English languages, and each page had options for discovery learning and tactile sensations in order to draw and keep children’s attention. For younger learners or children with developmental disorders, each page provided learning not only through auditory stimulation of someone reading to them, but through full sensory exploration learning per each page as well. Older learners with ASD or typically developing children were able to read the story but also to learn words in a second language, since on each page text was printed in both BHS and English languages as direct translations. These objectives were all aligned with the Bosnia–Herzegovina (BH) educational curriculum. Details about all experts involved in the creation of the book can be found on the last page of the e-book and credits page of the e-book.
Fig. 2
Children’s interaction with the printed book
×
“The Wolf in Love” e-book
According to the UNICEF-BH MICS Survey, only 56% of children in Bosnia and Herzegovina, of ages since birth through 5 years old, have at least 3 children’s books in the house (UNICEF-BH 2013). However, in the world today, digital media dominate children’s lives through watching TV, playing video games, reading stories through e-readers, tablets, and smartphones. In the USA, the percent of children with access to some type of smart mobile device at home (e.g. smartphone, tablet) has jumped from 52 to 75% from 2011 to 2013 (Holloway et al. 2013). Seventy-two percent of children ages 0–8 use a mobile device for some type of media activity such as playing games (63% of the time), watching videos, using apps, or reading books (only 30% of the time) (Holloway et al. 2013). In addition, a study by Palaiologou (2016) shows that in England, Luxemburg, Greece and Malta, about 80–90% of the 3–5 year-olds regularly watch TV programs or play games alone. Video games in particular provide an ideal multimedia tool that presents concepts in a way that are engaging, informal, and fun for children (Rieber and Noah 2008). Therefore, a potential benefit of educational video games is the value of generating engagement and motivation to learn for a student (Bourgonjon et al. 2010). New multimedia and interactive features of the animated electronic storybooks were found superior when it comes to vocabulary acquisition in kindergartens compared to static e-book version of the same text in Netherlands (Smeets and Bus 2014). Also, eye-tracking studies have demonstrated that children fixate longer on details in illustrations that are highlighted in the text (Verhallen and Bus 2011). Glenburg and Langston (1992) demonstrated that comprehension is better when the text is followed by illustration as pictures help to build mental models.
Table 1
The educational goals were to teach children to identify target objects (i.e. learn the vocabulary), 1:1 correspondence or learning to count objects from 1 to 10, to identify numbers 1–5, identify colours, and respond to inference questions
1:1 correspondence question, teaching children to count objects
3
Production responding, an inference question, where the child reads a short text and makes conclusions based on the comprehension of the story
Considering all this, an interactive e-book, based on the hard copy children book “The Wolf in Love” has been designed and developed (Hulusic and Pistoljevic 2016). Main aims of the e-book were to attract attention of a learner; to engage him/her by features additional to the text being read in two languages, e.g. audio feedback on interactive game objects (naming the object or making a related sound), and by providing a multi layered questions about the scene or story read. Some are selection responding, just pointing to the colour or one of the objects named in the scene, while others are abstract and inference questions like interpreting the feelings of a character in a scene. The overall educational goals of the game are teaching children to identify target objects (i.e. learn the vocabulary in native and foreign language), 1:1 correspondence or learning to count objects from 1 to 10, to identify numbers 1–5, identify colours, and respond to inference questions, Table 1. All these are developmentally appropriate educational goals for children in early childhood developmental stage. The difficulty of questions also changes across questions on each page to accommodate different levels of learners (young or a child with a disability vs. older typically developing child), Table 2.
In order to efficiently balance the game flow and avoid frustration and/or boredom, a user is allowed to skip to the next page without being forced to answer the higher-level questions. In addition, the order of predefined answers is randomised and multiple target stimuli for “point to” type of questions are provided. The cartoon-like, low-fidelity graphics in the e-book is used both because we wanted to preserve the same authored graphics used in the printed book, and since children are used to this graphics style from various children books and cartoons.
Table 2
Question difficulty levels shown per question and per page, e.g. difficulty levels of the three questions on the first page were 1, 2 and 3, while on the third page, they were 1, 3 and 3
Page
1
2
3
4
5
6
7
8
9
10
11
12
Levels
1 2 3
1 2 3
1 3 3
1 2 3
1 3 3
1 2 3
1 2 3
1 2 3
1 2 3
1 2 3
1 3 3
1 2 3
Several studies showed that when designing an educational serious game learning opportunities need to be integrated with the story content, a narrative needs to be built to support learning goals and encourage players to develop emotional connection with the characters, so that they will be motivated for learning (Kapp 2012; Whyte et al. 2015; Catalano et al. 2014). In developing this e-book, close attention was paid to these guidelines. We worked with the children story writer to create the main character, a wolf who falls in love and becomes a boy, while learning goals of recognising colours, numbers, letters, counting, vocabulary building, etc. were embedded within the story. Throughout the story, narrated by children in two languages, players can toggle the language of narration and game questions, and choose whether to hear a narrator or to rely only on the text, and whether to start the game portion on the page or to only explore embedded interactive functions within the story.
E-book development
The e-book was developed using several web technologies: HTML5, CSS3, JavaScript, jQuery and AJAX. These technologies allow for easy-to-use, real-time interaction through the web-based front end.
There are 13 scenes in the book, named scene1.html through scene13.html, Fig. 3. By default, when the e-book is started, scene1.html is loaded. All scenes are displayed through a single file, index.html. This is achieved through a navigation controller that loads a requested scene by switching the current html content with the content of the corresponding scene through AJAX, Fig. 4. At the same time, JavaScript method loadScene—resets the global variables was_corrected—used to indicate the correction; and questions_active—indicating if the questions were triggered. At this point, the event listeners take over and wait for the user to interact with the game.
Fig. 3
Three scenes from the e-book: scene 2 (left), scene 6 (middle) and scene 10 (right)
×
All html files have event listeners which are responsible for the page interaction. index.html controls the logic for the questions and answers, the language selection and for scene traversal, Fig. 4. Individual scene files are responsible for the events at each particular scene, e.g. playing the sounds, hiding and showing the elements, selecting the questions, and initiating the corrections. Each scene has a Less element preprocessor which defines visual appearance, sizes, positions and effects for that page. On top of that, there is an all.less file which defines the general page appearance including the language and question bar (Fig. 5), responsiveness, correction and animations. Everything in the application that can be translated has its key in the lang.js file. By clicking on the language selection button, the method translate() is called. The method translates all the system and local strings that are defined for the corresponding scene. The language can be selected at the opening scene, and also toggled throughout the game. This is an important feature that fosters learning a foreign language.
Fig. 4
A schematic view of the game structure
Fig. 5
A correction indicating the correct answer. Bottom box: the question button (top left), the language button (top right), the question and answers (middle), navigational buttons (middle left and right)
×
×
Each scene contains three questions, of which some have three provided answers. For each such question, only one of them, within its html element of the class answer, has class correct assigned. Questions that require interaction with the active scene elements, i.e. that use the “where is”, “point to” or similar antecedent, have their own logic implemented within their belonging scene. This logic is based on an additional listener that checks if the touched element has a class correct assigned to it. If so, the answer is considered as correct. For each correct answer, the “applause” sound is played, and one of the text responses (“Great job!”, “Excellent!”, “That’s correct!”) is randomly provided. In case of the incorrect answer, the correction is invoked. All the provided answers are shuffled using the method shuffle() in the index.html file.
Throughout the e-book, the game handles the score and displays it in the top left corner. The score is cumulative and updated for each displayed question through refresh_score JavaScript method within the index.htlm file in a “Correct answers / Total Questions Opened” form, where each is incremented accordingly, Fig. 6. At the same time, the game keeps track of all the opened and answered questions per level of difficulty. This breakdown is displayed at the end of the game, and provides teachers a better insight into the child’s performance and progress. It also allows teachers to concentrate more on the interaction with a child, rather than on recording the scores (per level of difficulty) manually. Finally, using this approach, there is no need for the interobserver agreement procedure obtained through reliability observations, requiring additional teacher to keep track of the score during game play.
Fig. 6
A cumulative score shown throughout the game in the top left corner (left) and the final scores—cumulative plus per difficulty level (right)
×
The correction for the incorrect answers is provided by either adding the blinking yellow frame to the correct answer (Fig. 5) for the questions with the provided answers, or by adding a red halo around the correct object (Fig. 7) for the questions with the “pointing to” antecedent.
Fig. 7
Corrections invoked after an element other than the target element had been selected
×
In each scene, there are a number of interactive elements. These elements, when touched, either make corresponding sound, such as tweeting for bird or howling for wolf, or are being named in the selected language. At the same time, they are either smoothly scaled up and down, so they gently pop up, or animated so they move across the scene. This is achieved by assigning the scale-animation class to the corresponding element, and defining the animation code in the corresponding .less file.
Finally, each scene has a bilingual narration, which is part of the storyline that introduces the child to the current scene. It is triggered by either clicking on the narrating character if in the scene (Fig. 8), or on any non-interactive element in the scene if the narrator is not present. Once triggered, it is both displayed visually on a predefined spot and played audibly. All the voice overs and the object naming audio files have been narrated by children.
Fig. 8
The visual appearance of the narration once triggered
Although the main target audience of the “Wolf in Love” series are children with developmental disorders, the e-book has been designed with all children in mind: younger typically developing learners or children with ASD and older learners with or without developmental disorders. This has been achieved through multilayer questions and bilingual implementation of the e-book, as discussed in “The Wolf in Love” e-book section. To validate its effectiveness on learning for both children with and without ASD, two user studies were conducted, as described in the following sections.
The three questions that went along with the text of the story on each page at different levels of difficulty were used to accommodate different levels of learners (a preschooler or a child with a disability, pre-reader vs. elementary school age typically developing child, a reader). Our educational goals in the studies were to teach children basic vocabulary, foreign language vocabulary and school readiness concepts, such as identifying target objects (i.e. learn the vocabulary), counting objects from 1 to 10, identifying numbers 1–5, colours, and responding to inference questions (i.e. comprehension of the text), Table 1. First question on each page was the easiest level question, a selection-based question with varied antecedents (i.e. “point to”, “find”, “where is” “touch”). Second question was either related to teaching children to count, using the scene from the book and objects in it, or the production responding, an inference or an abstract thinking question like whether someone was sad based upon the text on that page the child read/heard being read. The third question was always the production responding. These questions were based on the comprehension of the story, so for older children it would require reading skills. The overall goal of the interactive e-book was enriching the reading/listening to narration experience for children functioning on different levels and to provide an engaging learning environment to children with and without disabilities, but mainly multisensory needs of children with ASD.
User study 1: teaching children with ASD basic vocabulary and school readiness concepts
This user study focused only on the 36 questions embedded in the e-book. Following the reading, or listening to the text read on the page, student was able to select to engage in the game portion on the bottom of the screen (Fig. 3) and to respond to 3 questions per page base on the page visual and textual input. Following the mastery of the game, we assessed the transfer of the vocabulary and concepts learned through playing the game by asking children to use the vocabulary and concepts in their natural environments.
Participants
In this user study, only the game component of the interactive e-book was tested. Ten students participated, one girl and nine boys, with age ranging from 4 to 7 years old. All students were diagnosed with autism and/or other developmental delays with autistic elements. Children attended a specialised evidence-based model of instruction “EDUS—Education for All”, provided through public preschool program for children with developmental disorders. Children attended an evidence-based program that applied behaviour analysis to all aspects of teaching, for 5 hours daily, and all had individualised education plans and programs. Seven children attended a classroom with a student-to-teacher to teaching-assistant ratio of 10:1:1, due to the fact that they were more independent, had more verbal skills and were able to learn in a group. Three participants attended an 8:1:3 ratio classroom, where they required more intense language instruction in a small group or via one-to-one instruction. All students in EDUS programs were grouped according to their levels of verbal behaviour, so it would be safe to say that seven participants were speakers and listener, emergent readers and writer, while three were on a lower level of verbal behaviour and were emergent speakers and listeners, or simply had less advanced language and communication skills. Still, all children due to the ASD diagnosis had limited vocabularies and overall impaired language and communication skills compared to the same age typically developing peers. All instruction across all EDUS classrooms was done via learn units, teacher delivering instruction, reinforcement and corrections while continually taking data on all students responses and behaviours. Same rules were applied during the game playing while children sat at the computer table with the teacher who had a data collection sheet and read the questions from the game to the participants. Due to the age and level of functioning, children were not able to read the questions in the game to themselves. A teacher working with them was instructed to only get their attention to the game, read the questions and record student behaviours. He/she was not allowed to help students in any way or to prompt or explain any questions.
Design
Table 3
The instructions used in the questions, sorted per e-book page
Pg no.
Question 1 (Point to)
Question 2 (Count)
Question 3 (Identify / inference)
1
Find a bird on the branches
How many birds can you find? (5)
What is the color of the girl’s cowl? (red)
2
Find the wolf
How many spiders can you find? (6)
Who is the princess on the picture? (Rapunzel)
3
Where is the number 1/2/3/4/5?a
–
Do you see the boy and girl on the picture? (Y / N) On what are Hansel and Gretel sitting? (The Hundred Centipede)
4
Find the bucket of water
How many mice are there on the picture? (7)
Who is charged up? (Wolf)
5
Find the letter A/B/C a
–
What is the prince doing? (Waving) What is the color of Snow White’s bow? (Red)
6
Find the bush
How many flowers are there? (8)
Is the wolf sad? (Y / N)
7
Touch the mirror
How many stars are there? (9)
Can the good fairy fly? (Y / N)
8
Touch the keyboard
How many pins are there on the picture? (10)
Whose eyes are big? (Wolfie’s)
9
Touch the popcorn
How many bits of popcorn fell out? (4)
Does the wolf have his mouth full? (Y / N)
10
Touch the blue car
On how many books is the plane on? (5)
Where are the color pencils? (In the glass)
11
Touch the glasses
–
Of which color are the bows on the hearts? (Yellow) Whom does the chosen love? (Wolfie Wolf)
12
Touch the heart
How many large hearts are the on the picture? (1)
Is the boy wearing glasses? (Y / N)
The actual questions were differently formulated to reflect these instructions, e.g. the first question on the first page was: “Find the bird on the branches” where a correct behaviour was to touch on of the five birds. The second question was “How many birds can you find?”, with five birds on the page and three proposed answers: “4”, “5” and “6” in a random order. aThe value within the question was randomised
The participants played the game with a teacher reading the questions and taking data on the LUs (antecedent-behaviour-consequences). Only data on the game playing, not interacting with the e-book, were recorded for the purpose of this study. The instructions used in the data sheets during the data collection for all 36 questions in the game is shown in Table 3. Following the mastery, generalisation probes were conducted. They were conducted on non-screen media, that is, across different school environments and as responses to pictures in other children books. Criterion for mastery of the game was set at playing it once at a minimum of 90% accuracy, or 33 out of 36 correct responses to the game tasks. The data were looked at as total numbers of LUs (i.e. trials) required for mastering the game, that is, reaching the predetermined criterion level. Numbers of LUs to criterion were calculated as well, to tell us on average how many LUs children needed to master the game. Comprehension and vocabulary were only post-tested, to prevent that a repeated exposure to target words in tests could explain the outcomes, to avoid practice effect (Biemiller and Boote 2006; Verhallen and Bus 2010; Smeets and Bus 2012). During the generalisation probes, a total of 25 vocabulary questions were tested. They were conducted in children’s natural environment, during instruction and free play. Children were asked to point to or show target vocabulary from the game in their environment or in hard cover books and pictures in the classrooms. The questions/tasks from the generalisation probe data sheet are shown in Table 4.
Table 4
Generalisation probe: the questions / tasks, performed in pictures and/or books and in the children’s natural environment
No.
Question/task
No.
Question/task
1
Count 10 objects/manipulatives
14
PT “B”
2
PT RED
15
PT “C”
3
PT BLUE
16
PT boy
4
PT YELLOW
17
PT girl
5
PT “1”
18
PT bucket
6
PT “2”
19
PT Bush
7
PT “3”
20
PT mirror
8
PT “4”
21
PT Computer keyboard
9
PT “5”
22
PT popcorn
10
PT bird
23
PT glass
11
PT wolf
24
PT glasses
12.
PT millipede
25
PT heart
13
PT “A”
PT Point to
Interobserver agreement
Interobserver agreement or reliability observations were conducted with an independent observer for a total of 25% of all sessions of game playing, with a mean of 98.5% (ranging from 92 to 100%). While the teacher was reading the antecedents to the student and recording the students’ responses on a data sheet, an independent observer was doing the same on a separate sheet without any communication with the participant or the teacher. The interobserver agreement scores were calculated by dividing the total numbers of paired observer agreements by the total numbers of agreements plus disagreements and then multiplying that number by 100 (Cooper et al. 2007).
Results
The results of the user study 1, with 10 participants showed that the created software in native (BHS) language was completely clear and user-friendly for kids with developmental disorders, and that it is systematically and developmentally appropriately sequenced for learning, see Fig. 9. Additionally, it was found that children were able to generalise learned skills, through a transfer to new media or environments, by using the vocabulary in other hard cover books or by labelling 3D objects in their environment. Their teacher also reported that children were very motivated and enjoyed playing the game. For many of them, the game became a reinforcer, i.e. they asked to (re)play it. After an average of playing the game 4 times, all children showed a significant increase in correct responding and reached levels of over 72% accuracy, see Table 5. Six children mastered the game, reaching the 90% or more correct responses. Due to the objective limitations, such as time constraint because of the end of the school year, children’s absence and the fact that they needed more trials for the mastery, the other 4 participants did not complete the game. Participants G, H, I and J, were students on a lower level of verbal behaviour, from a higher ratio classroom, and therefore, naturally, their rate of learning was a bit slower compared to the children at the higher level of verbal behaviour (i.e. emergent reader and writers). For 3 out of the 6 who completed, we conducted the post-probes, maintenance probes to test for the transfer of learned concepts to the natural environment. All students demonstrated high transfer of vocabulary learned compared to other, non-computerised settings.
Fig. 9
The number of correct answers achieved per participants across the trials in the user study 1. Bellow the participants’ indicator the percentage of correct answers is presented. Last four participants did not achieve the mastery. Participant G was the only female (F) participant in the study
Table 5
The summary of the results of the user study 1
Participant
Age at the onset of the study
Verbal/ behavioural (VB) level
Number of times game played to mastery (36 LUs)
% correct achieved
Generalisation probes (25 LUs) (%)
IOA%
A
6.6
Listener speaker emerging reader and writer
3
97
100
100
B
4.5
Listener speaker emerging reader and writer
3
94
n/a
n/a
C
6.11
Listener speaker emerging reader and writer
5
92
n/a
92
D
4.10
Listener speaker emerging reader and writer
3
97
96
n/a
E
7.2
Listener speaker emerging reader and writer
2
97
n/a
100
F
5.3
Listener speaker emerging reader and writer
4
94
88
100
Ga
6. 8
Speaker and listener
3
72
n/a
n/a
Ha
4.2
Emerging speaker and listener
5
75
n/a
n/a
Ia
4.3
Speaker and listener
4
83
n/a
n/a
Ja
6.10
Emerging speaker and listener
4
72
n/a
n/a
aIndicates that participants G–J have not mastered the study (see “% correct achieved” column)
×
Testing such games with children with ASD is highly time and resource demanding, as the children played the game only once per day and testing a game with one child has to be performed by two teachers in an isolated environment and in a suitable time of the day that complies with the children’s daily teaching routines.
User study 2: teaching typically developing children foreign language vocabulary
The second user study was conducted with typically developing children, using the same content. This time, however, the e-book was used only in English, as well as all the verbal interactions between the experimenter and the participants. Upon the mastery of the game, the participants were assessed through the same generalisation probes (Table 4) as in the first user study, this time in English. Following the listening/reading the text on the page, children engaged in the game portion on the bottom of the screen (Fig. 3) and responded to 3 questions per page base on the page visual and textual input. Children played the game until the mastery of 90% or better correct responses to the questions. Following the mastery of the game, we assessed the transfer of the vocabulary and concepts learned through playing the game by asking children to use the vocabulary and concepts in their natural environments. Children were asked in English while playing or looking at other books, to show/point to a colour, object or a certain number, all concepts from the game.
Participants
The participant selection criteria for the study was that they are typically developing children, age four or more, without significant knowledge of English language. The criteria were set due to the fact that the children younger then four years old attempted the game, and due to frustration and very low beginning scores (i.e. 1 or 5 correct out of 36 on the first play) were unable to complete the game in the satisfying time period experimenters had. Therefore, seven children, four boys and three girls, ranging in age from 3.11 to 6.2 years old participated in the study. None of the children had any formal English education, and could not read the game questions on their own. Therefore, the questions were read to them by the instructors. The experimenters were also instructed not to help children either with the questions or with the language, just to read the questions, provide no explanations, and encourage the children to make a choice if they did not know the answer. Only in case of participant’s frustration or fatigue, experimenters were allowed to talk to children in their native tongue, still not helping with the game tasks.
Design
The design of the study was the same as described in Design section, keeping the criterion for mastery at 90% correctness. Although the score was kept by the game, the teacher was monitoring the child and recorded scores and acted as an independent observer, as described in Interobserver agreement section. The results from the generalisation probes are displayed in Table 6. These scores were kept manually by the experimenters.
Results
The results of the user study 2, with 7 participants in English, showed that the created software in foreign language was user-friendly for children and it is systematically and developmentally appropriately sequenced for learning a foreign language and concepts, Fig. 10. Additionally, it was found that children were able to generalise some of the learned skills, through a transfer to new media or environments, by using the vocabulary in other hard cover books or by labelling 3D objects in their environment. Interestingly, typically developing children showed more frustration when not knowing the answer or when “not winning”, while children with ASD were reinforced by just interacting with the e-book. This is developmentally appropriate, and “not knowing” or “loosing” is very difficult for children to accept at that young age. They just required more encouragement and after an average of playing the game 4 times, all children showed a significant increase in correct responding and reached criteria levels, see Table 6. All children but one, mastered the game with 90% or better accuracy, where one participant did not play after 86% accuracy due to the experimenter’s error in calculation of mastery level. All children were able to demonstrate generalisation of skills learned to natural setting for more than 50% of vocabulary/concepts. Interestingly, the youngest participant needed to play the game the most to master it and had the lowest results on the generalisation probe. This could be due to today’s children’s exposure to English language content in everyday situations through all different kinds of media. Anecdotally, one of the participants thought that “sad” was the same as “sat”, and answered “yes” to the question if the wolf was sad, thinking that “yes, he is sitting”. This shows that children even though formally not learning English, are accidentally picking it up through TV, music, video games, etc., and by not being provided full learn units, without correction and reinforcement, are acquiring wrong vocabulary.
Fig. 10
The number of correct answers achieved per participants across the trials in the user study 2. Bellow the participants’ indicator the percentage of correct answers is presented. Participant 4 did not achieve the mastery
Table 6
The summary of the results of the user study 2
Participant
Age at the onset of the study
Number of times game played to mastery (36 LUs)
% correct achieved
Generalisation probes (25 LUs) (%)
IOA%
1
5.1
4
100
68
n/a
2
4.4
4
100
92
100
3
3.11
5
92
56
100
4a
4.3
4
86
64
n/a
5
4.3
3
92
64
n/a
6
4.3
2
97
96
n/a
7
6.2
4
100
100
n/a
aIndicates that participant 4 has not mastered the study (see “% correct achieved” column)
×
Conclusions and future work
The interactive educational e-book presented in this paper provides a multisensory learning environment to children through game-like experience. At the same time children can read and listen to the narrated story in one of the two languages of their choice. Depending on the age and functioning level of a child, it provides an appropriate level of engagement: exploring each page of the book through interacting and watching animations; reading and analysing the story; and answering to suitable questions, designed using three levels of difficulty, e.g. selection-based questions, counting and production responding. The whole e-book is designed to retain children’s attention, utilising the latest research in education, interactive e-books and serious games. The educational goals were to teach children novel vocabulary, counting, identifying numbers and colours, and responding to inference questions both for children with developmental disorders and typically developing children.
The e-book contains multisensory stimulations, where extratextual explanations of word meanings are present, e.g. presenting a concept using the visual (image and text/word) and auditory (narration and sound) stimulation, and asking a child to click on a target object in the picture. This concept has been reported to have positive effects on word learning in kindergartens (Segers and Verhoeven 2003).
The results from the first study show that technology and computer game-based environments, when designed using findings from behaviour science, could be used for fast and effective skills development and knowledge acquisition. The six children with listener, speaker, emerging reader and writer verbal/behavioural level mastered all the questions within 3.3 trials on average. In addition, all three that were tested with the generalisation probes managed to transfer learned skills to new media or environments.
The second study, performed with typically developing children, showed that the same content can be used for learning foreign language basic vocabulary. However, if the score in the first trial was below 10, meaning that the level of English for the child is very low, it was more difficult to perform the test, as young children get easily frustrated when they are not succeeding, i.e. winning. Therefore, with the other 3 children, it was not possible to finish the study within the time frame experimenters had. Further studies should investigate what exact prerequisite skills are needed for children with ASD to play the game and learn successfully. Whether the children on a lower level of verbal behaviour can master the skills still needs to be tested and time needed for acquisition analysed. In addition, further studies with typically developing learners can make better recommendation what age or what baseline levels of knowledge are needed for motivated and successful learner. Comparing the results from these two user studies, we can see that younger typically developing children get frustrated and children with disabilities that are on a lower level of functioning in the domain of language and communication, may also struggle, needing more repetitions and game play.
Still, comparing the results of both studies, we can conclude that the e-book presented in this paper can provide preschool and school children with and without disabilities with appropriate educational software, to build up their early cognitive abilities and school readiness skills. Incorporating technology in preschool classrooms and especially inclusive classrooms, could benefit all types of learners. In addition, providing parents with such games as educational guides, would aid them help their children learn at home as well, especially for parents of children with disabilities. Multiple exemplars instruction is the best way of acquiring knowledge, so if the games followed early learning curricula, children would be immersed in learning via multiple exemplars automatically, learning in school, practising at home via video games, and generalising the learned easily (Greer et al. 2007). Also, game alone was designed to provide multiple exemplar instruction through different types of questions and presentations of the target stimuli.
While having score in the e-book has significant practical advantages, it is still not perfect and requires supervision for precise results. We have noticed that on few occasions a child told the teacher the correct answer, but while clicking on the target game object, a minor miss, e.g. clicking slightly of the object’s edge, has been scored as wrong by the game. This not only results in a wrong score, but can lead to frustration and demotivate the child. A solution to this could be to increase the touch screen object bounding area, if no more touchable objects are in its close proximity.
Throughout the study, we have also noticed that when counting objects on the screen, in one case, the correct answer was the smallest count among the options, leaving a very small space for the participant the make a mistake. This was only noticed through monitoring the gameplay, when a child was verbally responding with an incorrect, lower count than any of the options. This in turn made them count until they could find as many objects as the minimal provided count—in this case being the correct response. These scores were rated as wrong, and the game has been fixed.
Finally, even though the game was developed carefully using responsive design, the appearance on some screen aspect ratios is suboptimal. This is purely due to the fact that there are two distinct sections in the game—the top part with the e-book graphics, and the bottom part with the navigation, language selection and game questions. While this layout is found as suitable for most of the computer and tablet screens, for which it was initially designed, it requires improvements for mobile phones.
In the future, we would like to embed an assessment engine in the game, so that the data collection and descriptive analysis of learning outcomes are performed automatically in the background during the gameplay (Serrano-Laguna et al. 2017; Chaudy et al. 2014). Furthermore, we plan to extend the e-book with additional questions for teaching other skills and concepts within the same game environment. Despite being highly resource demanding, we would like to extend our user studies with children both with and without ASD, to provide us with better evidence for the game effectiveness on skills development and knowledge acquisition. Finally, we would like to develop a native mobile application that would overcome current limitations with content delivery and user experience.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
