3.1 Introduction
In Chap. 1, we made a case for the significance of touch for communication and suggested that developments in sensory digital technologies are bringing touch to the fore in ways that move digital communication beyond ‘ways of seeing’ to include new ‘ways of feeling’. We argued that this shift requires us to take new measure of digitally mediated touch, or ‘digital touch’, as a communicational resource, what it is and can be, how it is designed and imagined, and its communicative potentials and limitations.
In this chapter we build on this argument to map the current state-of-the-art digital touch technologies through an extensive review of the literature. We look beyond our everyday interaction with touch screens, to focus on emergent and semi-speculative touch technologies that ‘want us’ and ‘make’ us want to be able to touch and feel objects in new ways: from tangibles, wearables, haptics for virtual reality, through to the tactile internet of skin. We begin to map the complex terrain of digital touch by drawing attention to key developments in digital touch capacity; specifically contact and non-contact haptics. We then map the array of digital touch communication research in relation to different communicative relationships: human-human touch, human-robot/robot-human touch, and human-object touch. We use these conceptual distinctions to help to raise questions and start debates about the interlinked nature of social issues that arise across these different communication spaces and contexts, whilst acknowledging that there is inevitably some overlap of the technologies/ devices being developed and designed for use across these different contexts. Finally, the chapter, provides an overview of the scope, extent and findings of user studies to date, and in so doing, starts to document the resources for touch, the touch interactions supported and the kinds of touch communication practices that are being designed and starts to bring to the surface the social potentials and constraints of touch that are taken up by the designers of digital touch.
Anzeige
Primarily technologies being developed for digital touch communication involve some form of ‘haptics’. Haptics investigates “human-machine communication through the sense of touch in interactions where we can not only use our sense of touch for input, but also receive computer generated touch output.” (Huisman 2017, p. 391). Haptic technologies are used to convey human touch sensations (contact location, pressure, slip, vibration, temperature) and kinaesthetic perception (position, orientation, force). To do this, engineers and computer scientists need to measure human movement and sensations and match these to ‘haptic’ sensations that can be generated by various means and provided to the user. Haptic technologies can simulate various physical properties, such as the weight of an object, the feeling of friction, texture or resistance, or temperature.
There are two forms of haptic technologies: contact and non-contact. Contact haptics use specific input and output devices, such as data gloves or joysticks, for users to feel different, often mechanically generated, sensations, through force feedback or vibrating sensors (Smith and MacLean 2007; Bailenson et al. 2007; Takahashi et al. 2011), which can also be embedded into textiles and wearable devices. Responding to the importance of touch for human development and well-being, and recent empirical work suggesting that conveying emotion is possible through tactile interfaces that enable haptic communication between people (e.g. Réhman and Li 2010), a large tranche of research is taking place in the field of ‘affective haptics’ for interpersonal communication. Contact haptics are increasing the potential to generate physical sensations across a distance, e.g. using vibration, force feedback or mechanical motors integrated into various devices or materials, including wearable technologies. In so doing, this is changing the role of touch in communication – which is typically thought of as being co-located, with skin on skin, or skin on object – by extending the potential to use the ‘touch’ channel of communication remotely. Huisman (2017) offers a comprehensive review of haptic technology for social touch. Simulating touch is becoming an increasingly important consideration in virtual reality contexts, for enhancing feelings of immersion and fostering natural interactions through improving multisensory feedback, and for gestural interaction. A variety of approaches are being explored from haptic ‘digits’ (e.g. GoTouch VR) to haptic gloves (e.g. HaptX), combining touch feedback with touchless gestural interaction. HaptX gloves use microfluidics to create sensations of rigidity and softness through inflation creating indents into the skin to mimic skin depressions that result from holding or pressing physical objects in the world (https://www.youtube.com/watch?v=s-HAsxt9pV4). The potential applications for this ‘virtual’ touch are many including telerobotics, medical or military training, manufacturing design spaces, as well as entertainment and gaming.
Non-contact haptics involve the generation of physical touch sensations, or haptic feedback, without touching a physical object or device. Reverse electro-vibration is an augmented reality (AR) technology that generates a weak electric field around the user’s skin, allowing users to perceive the textures and contours of remote objects, without the use of gloves or specialised devices. Mid-air haptics (also called touch-less gesture tracking devices) uses ultrasound to make air pressure changes around the user’s hand, generating a physical sensation on the hand in mid-air, and usually combined with visual and/or auditory feedback. User studies suggest effective use in AR (Dzidek et al. 2018) their potential for use in VR/AR, music, robotics, automotive and teleoperation (Giordano et al. 2018).
Beyond haptics, advances in biosensing technologies, can generate information about the physical state of another, or of oneself, or embedded sensors can provide physical sensing of the wider environment. These technologies generate new ways to capture the quality of touch, or the environment that differently mediates touch interaction, and may even alter the very notion and possibilities of touch. For example, the Spanish avant-garde artist and cyborg activist Moon Ribas’ implanted a vibrating sensor in her arm, to detect real-time seismic activity from around the world. In relation to the Nepal earthquake she says, “It felt very weird, like I was there,” she says. “I feel connected to the people who suffer through an earthquake.” (Quito 2016).
Anzeige
In the context of robotics, developments predominantly focus on the hands for touching, with two different approaches. One is to develop just the necessary digits or qualities for the robot to perform a task it is designed for: this might mean only 2 or 3 finger grippers – a typical design for robots doing factory line picking and moving. The other approach is to develop robot hands as close to the human hand as possible, both in structure and function (e.g. Bianchi and Moscatelli 2016; Xu and Todorov 2016). However, much of this work concerns dexterity and movement, which although critical, are not so directly related to ‘touch’. Research on touch in robotics seeks to understand discrimination of the touch senses, both for humans and technically. For robots, the need to discriminate the meaning of a specific touch e.g. in sports or yoga, where tactile moving of a person’s body helps to gain the correct posture (DallaLibera et al. 2011), and the sensitivity of their touch movements is clearly important. Substantial progress has been made in the development of dexterity and sensitivity of robotic hands, in terms of their ability to detect objects and adjust, for example, the pressure with which to hold an object. Güler et al. (2014) explored how effectively robots can recognise substances by ‘feeling’ or ‘squeezing’, in comparison with using vision and touch, just vision or just touch. Findings suggest that vision alone and touch alone are similar in accuracy, and this improves when using both vision and touch. Alternatively, GelSight sensors (e.g. GelSight 2017) can be attached to robotic arms. These consist of transparent rubber, coated on one side with reflective metallic paint, that takes on the shape of an object when the surface is pressed against it. Current findings from both approaches suggest key effective features are rigidity or hardness, but more sensitivity is needed using, for example, vibration with tap or shake movements.
More extensively touch related, are skin-like technologies, where engineers and computer scientists aim to develop multisensorial material to cover large areas – similar to human skin – with a view to improving autonomous robots and enhancing biomimetic prosthetics, for example, Skinware (Youssefi et al. 2015) (foundation for Cyskin), e-skin (Hammock et al. 2013), iSkin (Coclite, Smart Artifical Skin, Weigel et al. 2015). E-skin comprises multimodal sensor skins that may be useful in: allowing robots to better sense their direct environment; soft prostheses that are capable of sensing contact, pressure or temperature; and as health-monitoring devices (Windmiller and Wang 2013). Hammock et al. (2013) place the start of e-skin to a 1974 prosthetic hand with discrete sensor feedback. E-skin then evolved to touch screens (1984, Hewlett Packard) to a material sensitive skin enabling a robotic arm to sense obstacles and avoid them (General Electric, 1985), to the 1990s when flexible electronic materials with large areas of force sensors were developed. In the early 2000s there was a rapid increase in the development of the integration of a wider array of sensors, using flexible, stretchable high-performance sensing capabilities, which aim to mimic human skin more closely, (e.g. Sekitani et al. 2014).
Drawing on these developments, iSkin integrates capacitive and resistive touch sensors that sense two levels of pressure, whether stretched or not, and supports multi and single touch (Weigel et al. 2015). It is flexible and stretchable and able to fit any area of the body, providing the opportunity for new types of on body devices, including finger-worn devices and extensions to conventional wearable devices. Studies of touch sensor recognition when worn in different areas of the body (the forearm, back of the hand and index finger) show high accuracy but identified challenges of low spatial resolution, issues of continuous pressure, and avoiding unintentional touch events. While e-skin technologies are progressing, this field of research is in its infancy, and few user studies or implementation of the technology ‘in the wild’ have been undertaken.
3.2 Human-Human Digitally Mediated Touch Communication
Globalisation, migration and changes in labour have led an increased need for communication at a distance, highlighting the changing place of touch in human to human communication. Enhanced portability and connectivity of the digital, has provided extensive changes to remote communication through video links, but more recent technical developments, bring new opportunities for new digitally mediated forms of social touch.
Much work in this area focuses on conveying or communicating emotion. Force feedback and vibrating sensors have been shown to be successful in conveying emotions, including angry, delighted, relaxed and happy (Smith and MacLean 2007, Bailenson et al. 2007). Other work shows how emotional experiences (e.g. hilarity) can be shared at a distance, through vibration triggered by either party watching the same movie (Takahashi et al. 2011). However, higher feelings of connectedness were found when combining speech and touch in a story telling scenario, using an upper arm touch device linked to a pressure sensitive casing on a mobile phone (Wang et al. 2012). These selected examples illustrate that force feedback or vibration can play a role in supporting mediated social or affective touch, specifically in terms of feelings of connectedness.
Textile sensors or wearable devices can also heighten and extend touch to communicate connection across distance, e.g. Ring∗U, a touch ring that provides vibrotactile feedback through an embedded eccentric mass vibration motor to ‘hug’ the wearer’s finger (Choi et al. 2014), or through stroking someone wearing digitally augmented clothing (Seeley 2011), or new ways of sensing the intention of, e.g. soft, touch from the way the hands move or the muscle activates through electromyography (Schirmer et al. 2011). A number of haptic jackets embedded with actuators enhance immersion in gaming or movie watching (e.g. Emojacket, Arafsha et al. 2012), and immersion in sports. For example, the ‘hugshirt’ or the ‘alert shirt’ (from We:eX) enable football fans to ‘feel’ what the players are feeling, e.g. heart rate changes or bump from a collision between players, through haptic feedback on the t-shirt. Here we see an example of how similar technologies are used for both individual ‘information’ or experience, and for connecting people.
An alternative focus has been on how technology might be exploited to realise the sense of physical/emotional warmth (Willemse 2015). For example ‘The Hug’ (DiSalvo et al. 2003), is an anthropomorphic cushion that communicates hugs by means of vibro-tactile and warm thermal feedback, ‘YourGloves’, ‘HotHands’, and ‘HotMits’ (Gooch and Watts 2010), support the feeling of holding hands over a distance, and ‘Huggy Pajama’ (Teh et al. 2008), reproduce hugs by means of inflatable air pockets and heating elements. This kind of warmth has been shown to enhance the idea of presence of ‘another’ (Gooch and Watts 2010).
While the field of affective haptics has shown how emergent technologies can differently connect people through touch sensations, and can be effective in achieving “a higher level of emotional immersion during media consumption, … communicating valence and arousal, and the emotions of happiness, sadness, anger and fear” (Eid and Osman 2016: 1), a number of challenges are also raised. The contextual impact of human interpretation of haptic communication is significant (Eid and Osman 2016), especially since mediated touch is dependent on the particular relationship between communicators, where in intimate situations touch can be seen as appropriate, but can generate discomfort in strangers (e.g. Smith and MacLean 2007; Rantala et al. 2013). The need for more insights into the effects of temperature-based stimuli and the role of other modalities in conjunction with the ‘touch’ itself is essential (Willemse et al. 2015), as well as the type of feedback that is most successful for conveying different emotions in different contexts e.g. warmth to reduce stress, haptic for social interaction (Huisman 2017). This is particularly important since attribution seems to form a large part of the mediated touch experience – where the haptic feedback need not necessarily feel ‘real’ but is attributed to the sender – another person or social actor – and thus takes on social significance (Huisman 2017).
We can see that various characteristics of touch form the basis of empirical research studies, such as: physical warmth (Willemse et al. 2015); notions of connectedness (Wang et al. 2012); different textures and wearables (Ebe and Umemuro 2015); developing meaning and ludic experience through conveying messages in gaming (Canat et al. 2016); or conveying different emotional feelings (Huisman 2017).
3.3 Human-Robot and Robot-Human Touch
Robots can be designed to look like humans, but the majority take other physical forms, the key factor being that they are programmed to automatically carry out a complex series of actions or tasks. While some robot designs include haptic sensors to provide the capacity for touch sensing, their automatic actions take them beyond ‘haptic devices’ per se. Nevertheless, touch is an important component in various areas of robotics research including affective and social contexts, and teleoperations.
3.3.1 Affective and Social Robot Touch
Since the 2000s, due to converging advances in technology and the changing social and economic landscapes of health, care and work, interest in ‘affective and social touch’ in robot-human and human-robot touch communication has grown. There are a number of research perspectives, including: research that seeks to understand human perceptions of robots, since this will impact the degree to which they are likely to be effective in affective or social communication; research which focuses on the mechanisms by which robot touch communication can elicit affective responses in humans; and the development and (sometimes) evaluation of robotic devices for promoting affective communication with humans. Some researchers classify interactions according to robot-initiated, human initiated or cooperative touch (Chen et al. 2011). In robot initiated touch the robot initiates contact with the human e.g. in care contexts (Mukai et al. 2010), in human initiated touch the human makes physical contact with the robot first e.g. with robot ‘pets’ (Yohanan and MacLean 2009), and in cooperative touch both are actively engaged in contact e.g. shaking hands (Shiomi et al. 2007). For technically mediated touch between robot and human, it is important to sensitively consider two key affective aspects of robotic interpersonal communication: being able to both convey emotion or meaning through touch, as well as interpret emotion or meaning through touch, as well as physical aspects, such as degree of pressure being exerted.
Given the importance of touch in social development and communication, there are assumed benefits for developing affective touch interaction between robotic agents and humans, requiring robot-initiated touch research. For example, Furuhashi et al. (2015) developed a robot that alerts the human of, for example, an incoming telephone call. When a call comes in the robot actively touches the person to alert them. User studies with adults showed challenges for the robot in negotiating obstacles in the room, detecting the location of, and recognising, the human.
For many researchers, the bi-directional connection between robots and humans is key. Rather than focusing on initiation of contact, work in this area includes developing prototype devices to explore the contact-expressive ability of the technology, while others engage more deeply in understanding human emotion and translating these characteristics in ways that can be emulated in robots. Erp and Toet (2015) argue that empathic communication is critical for social agents to improve social relations, and that social agents/robots with touch capabilities elicit more empathy and motivation to engage from humans e.g. in simulation, virtual patients able to touch back were treated more like humans than when not.
Affective touch prototypes have been developed to explore both human perception of affect and affective engagement with the robot device, and the effectiveness of the various haptic designs in conveying emotion. For example, pillows that respond to different kinds of stroking, pressure, and heat, or blankets embedded with electronics and computation, and which move and physically interact with people (e.g. Linköping 2004). However, no studies with these have been reported. In terms of robotic-touch and well-being, studies suggest benefits of pet robots in reducing stress and depression (Yohanan and MacLean 2011; Takayanagi et al. 2014), some of which specifically identify the role of touch – stroking, petting and hugging – in reducing systolic and diastolic blood pressure (Robinson et al. 2015) and mimicking hand massage experiences, which have been shown to release stress-relieving hormones (Remington 2002).
Research has shown that understanding human perceptions of robot communication is also critical in designing robots (Chen et al. 2011; Wullenkord et al. 2016). Motivated by the desire for robots to be as human-like as possible, Nie et al. (2012) investigated whether the temperature of a robot’s hand influences perception of the robot’s emotional warmth. Findings of a study with 39 participants suggest that experiences of physical warmth increased feelings of friendship and trust, but also raised the issue of exacerbating the ‘uncanny valley’ problem (i.e. the phenomenon whereby a too realistic humanoid robot arouses a sense of unease or revulsion in the person viewing it), and the need to take human expectations into consideration. Orefice et al. (2016) designed a robot hand with specific pressure points based on the human handshake and showed that gender and extroversion personality traits were interpreted, on the basis of firmness and movement of the hand during shaking, highlighting the communicative/ interpretative capacity of touch. (The ways in which digital touch is gendered is explored in relation to social norms in Chap. 4.)
A complimentary focus of research explores ways in which human touch can elicit changes in robot response (Martinez-Hernandez 2016). Here a model of touch is used to control robot facial expression, with five processing layers: sensation, perception, decision, action and worlds, which allow a human to change the robot’s (iCub) emotional state through tactile interaction. The researchers, experimenting with human to robot touch to assess the robot expression, found accurate recognition and response to actions like pinch/ stroke.
Longstanding ethical issues and the broadening of the ethical landscape beyond the human to include (in this case) the robot, are explored in Chap. 7.
3.3.2 Teleoperation
The field of teleoperation or telerobotics (operating a machine or robot from a distance) has a wide range of applications. Telerobots are used in the manufacturing industry for factory line picking and moving, for undertaking dangerous work, such as, bomb disposal or firefighting (Lawson et al. 2016), and in medicine, space, and marine contexts. Typically, a human operator controls a robot from a distance and receives feedback that informs whether the robot has followed instructions or completed the task.
As early as 1999 Fujita and Hashimoto demonstrated that technology can link together the actions of a robot arm remotely, so that moving the master arm will elicit the same movements in the robot arm. They also showed that users could feel their partner through force feedback, but not be able to see them. An example of training robots to recognise touch through learning from demonstration can be seen in firefighter training (Lawson et al. 2016). In this context, the robot nozzle operator needs to ‘understand’ human touch commands. Since force sensing resistors cannot be put all over the robot, Lawson et al., explore the use of LEAP motion sensors to recognise visual touch gestures, and use learning from demonstration (LfD) to teach the robot to recognise and react to various gestures. Similar methods are used with haptic gloves (e.g. HaptX, Shadow Robots and Syntouch), where leap motion sensors, attached to the glove, detect specific hand and digit movements and location, are used to elicit appropriate haptic feedback. However, in this case the gloves actually allow humans to sense what the robot is feeling (Aouf 2019), and are being designed for use in telerobotic contexts, such as, bomb disposal, space exploration and construction.
Another interesting area of ‘touch’ research concerns ‘body ownership transfer’ (Ogawa et al. 2012), where a teleoperator working with a robot can perceive the touch on the robot as if they themselves have been touched. In the teleoperator situation, only visual signals are received, and it is the visual event of the robot being touched that elicits the feeling of the operator being touched. Inoue et al. (2015) undertook a study with 8 adults to examine sense of body ownership, sense of agency, and mirror self-recognition ratings based on robot mobility and sensory-motor congruency, but their findings did not provide evidence for improved body ownership. In general, there seems to be little research to date that explores the concept of touch during robot training or robot manipulation e.g. understanding human perceptions of transferring their own notion of touch to that of the robot – whether it heightens awareness of the qualities of their touch, or its impact on their training or practice.
3.4 Human-Object Touch Communication
In this section, we look at how touch-based technologies (excluding robots) are enabling new communicative capacities between humans and physical or virtual objects.
3.4.1 Object/Textile Handling
With a predominance in online shopping, the textile industry is developing haptic based techniques for effectively conveying tactile qualities of materials (Perry et al. 2013). Touch is critical for customers and designers, who select clothing not only on the basis of what it looks like, but also how it feels, how the material falls and moves around the body. One approach to simulate or mimic texture and tactile elements of materials is to augment touch-like gestures e.g. a pinch gesture would lead to the material being visually scrunched (Orzechowski et al. 2011). Shoogleit, an application based on this idea, it was trialled with 218 university students (mostly female), who explored a chiffon dress or a man’s cotton shirt using the rotate (finger used to rotate the garment) and scrunch (pinch with visual image) capabilities (Cano et al. 2017), showed that the visual and touch were equal in their effect. Another approach is through ‘haptography’ (Culbertson et al. 2018), a combination of haptics and photography, where a stylus haptic device records textural data from different materials. This data is translated into various forms of haptic feedback, that enables different surfaces to be ‘felt’ through the stylus e.g. silk or canvas, although no user studies outside of the engineering lab are evident.
Other haptic technologies offer new opportunities for 3D object handling. ProbosVR, a tool akin to the phantom, uses a 3D interactive system that enables museum visitors to interact with scanned replicas of objects through ‘touching’ the objects using a joystick-like stylus, linked to related images, audio and video on an adjacent screen. Alternatively, devices like the vibrotactile glove enables users to feel 3D virtual objects in conjunction with seeing them (e.g. Martínez et al. 2016). Using a different technology – ultrahaptics – users can experience similar tactile interaction with visually projected (rather than physical) objects (e.g. Carter et al. 2013), where different textured surfaces can be recognised (Freeman et al. 2017).
Given the predominance of touch screens, electrovibration, a relatively new approach in the field of haptics, enables new user experiences that bring improved and increased kinds of touch experiences to ‘flat’ visualisations. Reverse electro-vibration enables physical objects to be augmented with different textures, creating artificial tactile sensations to almost any surface or object (REVEL, Bau et al. 2010). Typically, the research to date is taking place in museums, as well as entertainment and gaming contexts.
3.4.2 Education and Training
Research around touch and haptic technologies in education is somewhat disparate in terms of devices or systems, and topics or learning contexts. With children it has typically focused on those with tactile sensory loss or visual impairment, for rehabilitation purposes, or navigation (Patomäki et al. 2004). Research with mainstream learners has primarily focused on high school science, using the omni Phantom or joysticks, for example: sensing resistance between two molecules; simple machines (levers, gears, pulleys etc.); experiencing magnetic forces, mechanical forces; and for exploring viruses and nanoscale science; (ibid, pp. 2283). In mathematics the omni Phantom has been used to support dynamic geometry learning for 10 year olds (haptic with 3D visuals) (Güçler et al. 2013), trigonometry, using multimodal dynamic representations (abstract, visual and haptic) attaching haptic feedback to sine waves (Davis et al. 2017), and primary school geometry learning (Yiannoutsou, Johnson and Price 2018). With adults, the vibro-tactile glove has been explored to provide a haptic sensation of tracing the borders of 3D objects, but studies suggest that long training times are needed to develop the ability to perceive shapes (Martínez et al. 2016).
While research into the use of haptic technologies for school education is in its infancy, medical education and clinical contexts have adopted various ‘touch’ related technologies for enabling student practice of medical procedures, e.g. surgical, dental, and for improving efficiency and patient safety in surgical practice itself. For example, in dentistry, a stylus device can be used to feel over teeth to detect soft and hard surfaces of teeth to assess whether they need filling (Kuchenbecker et al. 2017). HAPtel extends this idea, using virtual reality in conjunction with the physical phantom device that represents a dental tool, to enable dental students to interact with a 3D mouth space to feel the different layers of a tooth when drilling (e.g. San Diego et al. 2012). Successful evaluation has led to extending this experience to practical restorative procedures. Haptics is also thought to be valuable in both practicing and undertaking minimally invasive surgical procedures, where the surgeon is separated from the patient and uses a robotic arm to do the operation. For example, work in the Haptic Intelligence Lab is exploring how to implement tactile feedback through instrument vibrations to reintroduce a sense of touch into procedures where touch is critical to the manipulations and actions being performed (e.g. Brown et al. 2017).
3.4.3 Disability and Rehabilitation
For some people living with a range of disabilities, digital touch capacities can enhance their quality of life, for example, through new rehabilitation systems, tactile applications for the blind, and e-skin for prosthetic purposes.
The tactile sense can sometimes be seen as a substitute for other sensory inputs, particularly for the visually impaired. While developing techniques that exploit the tactile sense for the visually impaired is not new (e.g. Braille Warren 1978; tactile maps or graphics, Sheppard and Aldrich 2000), developments in haptics and touch screen interfaces offer alternative ways of exploiting the tactile sense for this group, particularly for navigation, access to information and spatial awareness.
Although technologies initially used sonic feedback to facilitate mobility and navigation (e.g. Heyes 1983, miniguide), tactile stimulators provide vibration or tapping to guide the person (Ross and Blasch 2000), and the PHANToM can provide spatial information in a virtual environment (e.g. Magnusson et al. 2002). More recently the PHANToM has been used in classroom settings in the form of multimodal games for supporting visually impaired children’s engagement with 3D objects tracing pathways/shapes (Patomäki et al. 2004), learning geometry (Yiannoutsou et al. 2018) and learning of electric circuits (Pietrzak et al. 2007).
Combined with touch screen interfaces work has focused on kinetic tactile displays which enable ‘active touch’ i.e. feedback coupled with location. Several of these interfaces use actuators, motors and pressure (e.g. Velazquez et al. 2008). However, these methods have disadvantages in terms of resolution and range, and cost. In contrast, TelsaTouch uses a conductive layer to provide tactile sensation to moving fingers on touch screens (Xu et al. 2011). Findings from application studies showed that difficulties with navigation need further work, the subtlety of dots in Braille was hard to perceive, but solid shapes were easier to recognise. Since the information processing capabilities of the tactile sense are lower than vision, haptic alone has been shown not to always be sufficient (Levesque 2005), leading designs to augment tactile displays with audio e.g. Talking Tactile Tablet (Wells and Landau 2003), or thermal screen, with varied temperature generated by the embedded LED bulbs, that allows a blind person to paint colourful pictures on the tablet (Kos et al. 2016).
Moving away from screens, haptics in the form of clothing and textiles, vibrotactile gloves, VR and e-skin devices are being researched for rehabilitation purposes. Clothing and textiles designed to convey information about the wearer can be used for navigation, e.g. through vibration in shoes (Rowley 2016) or to correct posture through directional feedback in clothing indicating which way to rotate e.g. the ankle, with frequency and vibration being used to convey how far off the correct position the wearer is (Van Dongen 2017).
Interactive experiences that aim to foster rehabilitation are being developed in VR, using controllers or vibrotactile gloves. A review (Rose et al. 2018) of 18 papers to explore research findings related to enjoyment, rehabilitation routine and health outcomes, and the role of haptic feedback on VR immersion and performance, showed that haptic controllers served to increase movement accuracy, while gloves decreased movement velocity. Use of the vibrotactile glove has shown slight improvements in muscle strength and hand movement for post stroke patients (Hsiao-Ching et al. 2017). A review of VR (joysticks, Kinect, but not headsets) for rehabilitation for children with cerebral palsy, shows improvement in balance and motor skills (alongside traditional rehabilitation methods) (Ravi et al. 2017).
More recently, an e-skin sensing device with a view to use in prosthetics, was developed using electrotactile stimulation. Testing with 8 healthy participants, to see if they could recognise shape, position and direction of mechanical stimuli presented on e-skin, showed good performance levels, but highlighted challenges of computational complexity in successfully integrating e-skin into prosthetic devices (Franceschi et al. 2016).
From this overview, we can see an increasing number of contexts and applications are employing haptic or sensor technologies to convey information to users in different social and communication contexts, including object or textile handling in museums and commerce, education and medical training, and to alleviate issues of disability, through rehabilitation, prosthetics or enhanced forms of interaction.
3.5 Conclusion
This chapter has provided a foray into the landscape of digital touch technologies. Technological development in this area is somewhat in its infancy but it is bringing a diverse set of techniques and engineering capacities, as well as various approaches to informing or underpinning designs and applications, depending on the area of use. As we have seen, digital touch technologies are being developed for health and well-being, education, personal relationships, industry and work contexts, each demanding different consideration. While some touch (haptic) technologies have been integrated into, for example, medical training, a significant proportion are in the early stages of research development, perhaps more at a ‘proof of concept’ stage. This is especially true for VR (Stone 2001, 2019), but also the many challenges facing robot interaction with humans, not only with respect to human interpretation of robot intention and robot understanding and navigation, but also in terms of significant ethical issues. From the mapping of this landscape we now turn to the social and cultural questions, issues and considerations raised by digital touch.
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.