Developing Kaspar: A Humanoid Robot for Children with Autism

The first Kaspar robot (K1, Fig. 4) designed in 2005 by M. Blow et al. was intended to be a Human–Robot Interaction platform for a short term research project [7]. Since Kaspar has a human-like face with realistic but simplified features (i.e. it has a nose, eyes, a mouth etc. but no facial hair or skin colouration), the robot was soon adopted to study how such robots could be used as therapeutic devices for children with ASD [6]. At that time, the Kaspar robot presented a leap in the field of assistive robotics for children with ASD as the Robota robot (Fig. 2), that had primarily been used in this field before, only had 5 DOF, which significantly limited the gestures, body movements and postures that the robot could adopt, did not possess and expressive face, and was quite small (doll-sized). By contrast the K1 Kaspar robot possessed 16 DOF and measured approximately 46cm in height by 36cm width and 36cm depth with an expressive face capable of producing simple but realistic expressions. The construction of the K1 robot was based around a shop window dummy body of a 2-year old child for the main chassis with the robotic elements being constructed from off the shelf RC servos and hand fabricated metal parts. The servos used for the robot were relatively inexpensive as one of the design principles for the robot was to produce it for less than 2000 euros. The head of the robot in particular used micro servos for the eyes because of the compact nature of the space required to fit the components [7]. The K1 robot was powered by a 12VDC low voltage lead acid gel battery with a run time of approximately 2 h from a 4 h charge time. This first version of the Kaspar robot was used for a number of activities and games that encouraged skills such as turn taking to the exploration of facial expressions [13‐16]. Further to this the robot was also used to explore the possibility of conducting robot-mediated interviews with children [17‐20] and to explore interaction dynamics and gestures in human–humanoid drumming experiments [21, 22] The second implementation of the Kaspar robot K1-L was built in 2006 and was similar in construction to the K1 version but was a much larger robot using a 6-year old child dummy as a chassis measuring approximately 123cm in height by 30 cm wide and 57 cm deep. This version of Kaspar was mounted on a desktop PC and was equipped with an extra DOF in each arm, providing the robot with a total of 18 DOF. Because this version of the Kaspar robot was primarily intended as an early software development platform for the Robotcub project [23] it was also furnished with additional sensing capabilities including a laser depth camera and joint feedback sensors. Rather than being used for children with ASD the purpose of this version of Kaspar was for cognitive and AI software developments, testing and HRI studies [24]. After this version of Kaspar all future iterations of the robot were designed to be used specifically with children with ASD in schools or home environments and as such were designed to be easily transported and set up.

The K1 robot was very successful in its ability to attract and maintain the attention and interest of many children with ASD who had interacted with the robot. It provided more advanced interaction possibilities than the previously used Robota robot. Thus, a second implementation of Kaspar (K2, Fig. 5) was developed in 2006 by M. Walters to facilitate further research investigating the potential applications using such a robot with children with ASD [25]. The K2 robot possessed a specification similar to the K1 robot—with 16 DOF and a comparable overall size, measuring approximately 50 cm in height by 36 cm width and 36 cm depth. The main upgrades to K2 were simplifications to the hardware in the robot’s head to make the robot more robust and the inclusion of colour cameras in the eyes as an upgrade to the black and white cameras of K1. One of the most notable developments implemented on K2 was the integration of a Nintendo wii [26] remote to facilitate a fully autonomous collaborative game for children with ASD to play dyadic and triadic games [27, 28]. The game used a small monitor to provide visual feedback and wii remotes strapped to the arms of each player. In order to earn rewards in the game the participants had to collaborate and work together. Using this setup two studies were conducted, a dyadic study [27], evaluating how children play with a human player compared to playing with a humanoid robot, and a triadic long-term study [28] where the children played triadically with the Kaspar robot with all players (including Kaspar) having an equal role in the game. Pre- and post- assessments where pairs of children played the game with each other, without the robot, showed improvements in the children’s collaborative skills. However, there were some important lessons learned from this study that were taken forward to future developments of the Kaspar platform. Because Kaspar was operating as a fully autonomous system in this study, it greatly limited the type of children that could be worked with. Previously, the studies conducted with Kaspar had focused on working with children on the lower functioning end of the ASD spectrum. However, the constrained nature of a fully autonomous setup made it impossible to work with low functioning children which requires much more flexibility of adjusting the games on the fly. Also, this setup required the robot’s external sensors, i.e. the wii remotes, to be attached to the children’s arms which is not ideal because these devices can move and become loose. They also have the capacity to distract the children, and for some children with ASD it can be uncomfortable to be required to wear extra sensors/clothing/devices on their bodies. Taking these factors into consideration all future developments to the Kaspar platform focused on semi-automation rather than full automation because this would make the system more flexible as well as useful to a wider range of users. Further to this, relying on sensors that the children would have to wear was avoided in the future.

Driven by a European project called ROBOSKIN, in 2009 the third version of the Kaspar robot was constructed by M. Walters (K3, Fig. 6). Identical in size to K2, K3 measured approximately 50 cm in height by 36 cm width and 36 cm depth, with an extra DOF giving the robot 17 DOF in total. The aptly named ROBOSKIN project was focused on developing robotic skin capable of facilitating tactile interaction with robots [29, 30]. The K3 was largely based on the K2 but included a number of new features based on what had been learnt from previous versions of the robot. Unsurprisingly the most notable feature added to the K3 was the addition of tactile skin patches strategically placed on the robots feet, hands and chest to enable tactile interaction and to assist children learn about what is and is not appropriate tactile interaction. The tactile sensors on the robot were called ROBOSKIN and used distributed pressure sensors based on capacitive technology. The transducer consists of a soft dielectric sandwiched by electrodes. When force is applied to the sensor patch the distance between the electrodes change, causing the capacitance to change accordingly. The ROBOSKIN system in particular was constructed with a number of tactile elements (taxels) geometrically organized in interconnected modules of triangular shape. The flexible PCB was covered by a layer of silicone foam and acted as a deformable dielectric. The additional DOF integrated into the robot was placed in the torso and enabled the robot to turn left and right creating the possibility of more complex gestures. Having this additional DOF allowed the robot to turn away, e.g. when a child had hit the robot. Since Kaspar is not a mobile robot, this was an important additional feature. It allowed us to teach children with ASD about socially appropriate tactile interaction [31]. Children with ASD may either crave or avoid tactile interaction (named hyper- and hyposensitivity), so this facilitated the development of a range of tactile interaction scenarios. In addition to the ROBOSKIN and the torso, a speaker was integrated into K3 which assisted in creating a stronger effect of the robot itself speaking as opposed to the sound coming from a slightly displaced location, e.g. a laptop or additional speaker. Studies making use of the ROBOSKIN installed on Kaspar showed that the sensors could be useful in facilitating robot assisted play and had the potential to expand the repertoire activities and functions that the robot could perform in an assistive capacity for children with ASD [32, 32‐37]. Because of the success of the upgrades to the K3 robot in terms of their ability to facilitate more complex and useful scenarios in a more realistic way, all future versions of the Kaspar robot would include tactile sensors, an on-board speaker and the additional DOF to enable the torso movement.

Based on the positive results of the previous studies with the Kaspar robots since 2005, indicating that robots such as Kaspar could assist in helping children with ASD e.g. develop their turn taking skills, collaborative skills and tactile social interaction skills [38], the first small production run of 7 Kaspar robots was designed and developed by Merlin Systems Corporation Limited in 2011 (K4, Fig. 7), contracted by the University of Hertfordshire. The purpose of these robots was to enable more research to be conducted into how the robot could be used to help children with ASD, but also for the Kaspar robot to be used outside of the immediate research team with teachers in schools. The K4 robot measured approximately 57 cm in height by 34 cm width and 36 cm depth. Similar to the specification of the K3 the K4 had 17 DOF and included a speaker in the body of the robot. The K4 robot was inspired by the K2 and K3 robots but was totally redesigned. Rather than using a dummy body as a chassis like all previous versions, this version was engineered and fabricated from scratch using plastic and metal parts. Because the tactile sensors were used to such positive effect in the K3 robot [37], the K4 robots also included 10 tactile sensors. However, the sensors integrated into the robot were Force Sensing Resistor (FSR) sensors and were simpler, but much more reliable and cheaper than the initial prototype ROBOSKIN sensors available for K3. Using these FSR sensors allowed the component cost of the robot to be kept below 2000 euros whilst still providing adequate functionality for tactile interaction. To utilise these sensors the software that runs the Kaspar robot required further development. The initial integration of the FSR sensors was conducted by Barbadillo [39] but was later refined by O. Novanda to filter the electrical noise from the sensors. Another change on the K4 robot was the type of servo used. The K4 was developed using the smart Dynamixel AX-12 servos where previous generations used much more basic servos. Because these servos were far superior to the servos used on previous robots, the K1, K2 and K3 robots were all upgraded with these servos too, to increase robustness and accuracy of the robot’s joints. The K4 robots were also equipped with a small on-board PC complete with Wi-Fi and Ethernet connectivity. However, because the software that these robots came with was not sufficiently reliable or user friendly it was later replaced with a USB serial connection that had been used with previous versions. A number of studies were conducted with these robots and they allowed research to be conducted outside of the immediate research team in the university because of the number of robots produced [40, 41].

The K5 robot (Fig. 8) was designed from scratch by L. J. Wood and M. Walters in 2014, with 20 robots being produced. The K5 was slightly bigger than previous versions to accommodate the new features and as such it measures approximately 56cm in height by 34 cm width and 40 cm depth. Lessons learned from all previous versions of the robot along with modern design and manufacturing methods were used to radically re-design the platform. Because a production run of 20 robots was planned, the design of the robot also needed to suit this level of manufacturing. Therefore CAD design, 3D printing, laser cutting and vacuum forming were used. This was the first version of Kaspar that was suitable to be used by parents and teachers or therapists independently without a researcher present.

Since 2014 a number of hardware and software upgrades have been made to the K5 platform to improve the functionality and usability of the robot constituting the K5.5 robot (Fig. 9). Part of the upgrade to Kaspar was the integration of an RFID reader which is ideal for the Kaspar platform as it is an inexpensive and reliable technology. Currently the RFID reader is used to switch between game modalities on the robot without having to use the PC, but there are plans to use this technology in the future to enable the robot to detect tagged toys and build games around these toys. This technology is well suited to creating a level of autonomy within the robot due to its robustness. In addition to the RFID technology the hands of K5.5 included upgraded hands with strong neodymium magnets to enable the robot to hold objects placed in its hands. This upgrade has already had an impact in a nursery school where the Kaspar robot has been used with magnetic accessories including a fork, spoon, hairbrush and toothbrush. The staff from the nursery have reported that this feature has been useful for encouraging some children with ASD to eat by simulating eating with the magnetic spoon and fork, and it can be used to teach about personal hygiene, e.g. brushing the teeth or combing the hair. Those upgrades on the K5.5 robot have all been driven by the user requirements of assisting children with ASD, and were in fact based on suggestions by teachers, and as such have been well received.

The Sense-Think-Act architecture on Kaspar is a network-based platform independent architecture that has been implemented in a number of different programming languages (C sharp .Net and Java) and is capable of being distributed over a number of different processors in order to reduce the total computational cost. Since we have created the architecture on the basis of the IrisTK toolkit, the architecture has the capacity to operate on a number of different machines. In this instance, several IrisSystems are connected to a central broker (IrisBroker), which relays events to all connected systems. In each machine the events are serialised in standard JSON data packets and sent over TCP/IP. Using this method, it is also possible to connect modules that are implemented in other programming languages (Fig. 12). The architecture includes three standalone layers fully interconnected via a TCP/IP network (Fig. 11). Each layer has a number of modules that process either the sensory data captured by sensors/hardware or the high-level information that is distributed to the network as “events” in the standard JSON data packets. The layers and modules are fully interconnected and have the capacity to send and receive “Events” via the Broker over the network (Fig. 13). Thanks to the architecture’s modularity and network structure, the system is capable of running on multiple devices which facilitates handling of the overall processing cycles for real-time applications, if required. One of the primary benefits to this architecture is the potential for scalability allowing us to easily extend the architecture by adding new sensors/hardware devices and also new modules to the system. The architecture operates by collecting the sensory data and extracting high-level information then streams the corresponding “events” as JSON data packets to the networks (Sense Layer). The central layer receives the JSON packets and evaluates which reactive behaviour is the most appropriate for the current situation taking into account the interaction status and high-level information, and then streams an action “event” (behaviour name) to the network and asks the robot to display that behaviour (Thinks Layer). The Act layer receives the action event from the network and displays the behaviour on the permission of operator and returns the feedback/monitor “event” to the network to confirm that performing the action has been completed (see [47] for further details). Since the architecture communicates the events in JSON packets it is ideal for real-time HRI where the data communication is extremely quick with minimal lag time. Although the Sense-Think-Act architecture is fully interconnected meaning that the modules have the capacity to receive all the distributed events over the network. In order to reduce the computational costs, in each layer there is the possibility to subscribe only to those events that are necessary for that layer and dismiss all the other events (Fig. 13 black arrows).

The sense layer includes a number of sensors which sense the environment and the associated perception modules that we have developed to interpret the sensory information in order to extract “events” occurring in the CRIs. We have chosen the sensors and developed the associated perception modules based on the requirements of the VPT games that we have been developing for the Kaspar robot. The core technical requirements to facilitate these games are object recognition and 3D orientation tracking.

The Think layer is functioning as the brain of the architecture and as such receives all of the event streams from the Sense and Act layers via the network (IrisTK Broker) in real time and decides how to handle this information in order to make an appropriate decision for the robot’s action. In the decision-making process it considers the game modality (which game the robot is currently in), the status of the game (the progress of the child in the game), and the previous action shown by Kaspar. Figure 16 illustrates an example of the implementation of the games scenario in the Think layer. The interaction flow has an initial starting point for example state “C” in Fig. 16. The starting state could be an action such as the robot greeting the child or giving an introduction on the game to the child or whatever actions which encourage the child to engage and start the interaction with the robot. The Think layer activates the starting state once it has receives the relevant event/signal from the Sense layer for example a signal which shows child intention [50]. Using two main commands (“go to” and “return”), the interaction flow goes back and forth in the states to control the robot’s actions to procced with the game with the child in a way that will help the child to succeed in the game. For example the successive states could be for triggering robot’s action to provide positive/negative feedback to the child. The last states of the interaction flow could be to trigger the robot’s action to give a signal to show the game has been successfully completed or to ask the child to repeat the game if they would like to. In addition to the type of robot action in the interaction with the child, the dynamic of the robot’s action (such as gaze and attention) is controlled following the work presented in [51]. As shown in the Think layer, following the arrows, the states of the interaction flow that will be triggered are in the following order: “C (D>E), A, F, B”. The output of the Think layer is the name of the behaviour/action that system wants Kaspar to display and it is being sent directly to the Act layer. In fact, to keep the human operator in the robot’s control loop, the Think layer firstly shows the name of the behaviour to the human operator (on the screen), and Kaspar displays that behaviour only on the approval of the operator. Because a number of VPT games were implemented as different interaction flows, we have put all the flows in a single interaction Flow and allowed the user to choose the game which is preferable (Fig. 17). Therefore, prior to starting the game with the children the operator specifies the game number by scanning an RFID card to the system and afterwards the architectures will switch to that section of the Flow which is relevant to that game.

The final layer of the architecture is a reactive (act) system which has been developed to provide Kaspar’s control signals in order to display different behaviours on the robot. to the IrisTK Broker and receives all the events however we have subscribed only to the Think layer which means that the reactive system only listens to the events that streamed from the Think layer. Similar to other layers, the reactive layer is also connected. The Kaspar reactive system (Fig. 16) has several pre-programmed behaviours stored as different external files that are typically used for generic play sessions and include various postures, hand waving, drumming on a tambourine that is placed on its legs and singing children’s rhymes. Each behaviour file includes the names of the sequences that are required to generate that behaviour. Each sequence file includes 22 motor position values to control Kaspar’s servos, and also the name of the voice files are to be played by Kaspar. With the previous Kaspar control architecture we were able to activate these behaviours by pressing a buttons either from a keypad or from the software interface. However in the semi-autonomous version, the Think layer will decide and activate a behaviour by sending an action event to the Act layer via the Broker. The Act layer has a sequence-player method that receives the name of the behaviour and plays the corresponding behaviour sequence. Figure 18 illustrates the Kaspar GUI for the reactive system which is connected to the architecture via the Broker. As shown there are two boxes (red, green) on the bottom-right corner of the GUI. The red box displays the behaviour that is estimated by the deliberative system according to the “perceptual information” provided by the Sense layer, and the green box displays the name of the correct behaviour that system estimates based on the “interaction status” and the “Interaction Flow”. These boxes will be shown on the GUI and the human operator has to make the final decision for the robot’s behaviour. The operator must give the final permission to the robot to display the behaviour presented in the red box or can override the robot’s behaviour and ask robot to display the behaviour presented in the green box.