Wearable Haptic Devices for Realistic Scenario Applications

How do you perceive the world? When someone asks this question, the simplest and most logical answer might be “through the information coming from our senses”. Since ancient times, numerous theories have emerged regarding the role of the senses and the perception of reality. It should come as no surprise how traumatic the loss of one or more sensory modalities can be for our life experience. The aim of this book is to assist those who have suffered such trauma by leveraging one of our primary intrinsic recovery mechanisms for sensory loss: neural plasticity, through the use of wearable haptic systems. This first chapter aims to present a concise overview of the theory of perception through the ages, explore how our bodies respond to sensory loss via neural plasticity, and discuss the related haptic interfaces designed to address these issues. Special attention is given to two disabilities intensively explored in the state of the art: blindness and upper limb loss.

This chapter serves as the cornerstone of the book, focusing on the identification and analysis of user requirements critical for the design of wearable feedback systems and navigation aids. Through a user-centred approach, the chapter delves into the specific needs and preferences of visually impaired individuals as well as those with upper limb amputations. These insights are essential for developing effective and intuitive technological solutions. The findings presented here lay the groundwork for the subsequent development and implementation of wearable haptic feedback devices and indoor navigation systems, detailed in later chapters. By prioritizing user needs from the outset, this chapter aims to ensure that the resulting technologies are not only innovative but also practical and user-friendly, addressing the unique challenges faced by both visually impaired users and amputees.

This pilot study aims to determine which tactile sensory feedback, when combined with a soft robotic prosthetic hand, provides the most advantages for person with limb loss performing everyday tasks. Five volunteers with trans-radial amputations used the new SoftHand Pro (SHP) prosthetic hand to test a study protocol that evaluated single and combined feedback modalities while accomplishing various tasks. The tactile cues provided included contact information, grasping force, degree of hand opening, and combinations thereof. To transfer such feedback, different wearable systems were employed, utilizing either vibrotactile or force stimulation in a non-invasive, modality-matching approach. The results indicate a preference among amputees for single, rather than combined, feedback modalities. The choice of appropriate haptic feedback appears to be subject- and task-specific. Furthermore, in alignment with participants’ feedback, force feedback with adequate granularity and clarity emerged as potentially the most valuable feedback type among those tested. Lastly, the study suggests that users prefer a prosthesis equipped with precise and clear feedback mechanisms.

This work introduces an enhanced version of the wearable haptic device, Clenching Upper-Limb Force Feedback (CUFF). This new design offers a smaller and more efficient device capable of delivering both skin stretch and grasping force feedback to the user’s arm, making it highly suitable for prosthetic applications. In our research, we conducted thorough physical and perceptual tests with eleven able-bodied participants and carried out two key experiments involving a total of 32 participants. The CUFF device was used alongside the Pisa/IIT SoftHand robotic hand in both experiments. The first experiment focused on a discrimination task where participants differentiated between pairs of cylinders based on their size and perceived softness. In the second experiment, participants used the robotic hand to grasp various objects. After these tasks, participants provided feedback on their experiences with the device. The results from these experiments, along with participants’ subjective evaluations, highlighted the effectiveness of the CUFF device. Thanks to its versatile design and functionality, this device shows great promise for a wide range of applications, including teleoperation, guidance, rehabilitation, and prosthetics.

In this work, we propose a novel indoor navigation system based on wearable haptic technologies, aimed at increasing the autonomy and improving the quality of life for blind individuals. Despite promising research outcomes in the field of technological travel aids, these solutions have seen limited acceptance in real-world applications, due in part to the insufficient involvement of end-users in the conceptual and design phases. Our proposed system was developed with continuous feedback from visually impaired persons. It consists of an RGB-D camera, a processing unit that computes visual information for obstacle avoidance, and the CUFF device, which provides normal and tangential force cues for guidance in unfamiliar indoor environments. Experiments with blindfolded subjects and visually impaired participants demonstrate that our system could be an effective support during indoor navigation and a viable tool for training blind individuals in the use of travel aids.

In this work, we present the Clenching Upper-Limb Force Feedback Device (CUFF), integrated with the SoftHand Pro robotic hand. Grasping an object is a complex action that relies on sensory feedback. While prosthetic hands assist with mechanical performance, they often lack adequate sensory feedback. Providing feedback about a prosthetic hand’s grasp force is crucial for individuals with limb loss. We tested the CUFF system, controlled via myoelectrics of the forearm muscles, with five participants with limb loss and nineteen able-bodied participants. They completed a constrained grasping task (with and without feedback) requiring modulation of the grasp to reach a target force, with vision and hearing significantly limited. Data were analyzed using generalized linear models and Functional Principal Component Analysis (fPCA). CUFF feedback improved the able-bodied participants’ ability to achieve the target grasp force and enhanced grasp precision for participants with limb loss who typically use body-powered prostheses, as well as for a subset of able-bodied participants. Further testing is needed to determine if CUFF feedback can accelerate the mastery of myoelectric control or benefit specific patient sub-groups.

This work presents a soft pneumatic feedback system designed with integrability and wearability as primary concerns. Modern active prostheses can help recover some motor functions lost with hand amputation, but they typically fail to restore sensory abilities, forcing individuals to rely predominantly on visual feedback. Few solutions have been effectively integrated into prostheses for real-world use. At the system’s core, two soft pneumatic actuators are placed in contact with the subject’s skin and inflated to provide pressure stimuli, representing the force exerted during hand grasping. We report on the design and characterization of the system, including behavioural experiments with able-bodied participants and one prosthesis user. Results from psychophysical, dexterity, and usability tests show that the system has the potential to restore sensory feedback in hand amputees and can be a useful tool for enabling correct modulation of force during grasping and manipulation tasks.

In this chapter, we focus on the essential paradigm of modality matching for haptic feedback. Modality matching refers to the natural association between a sensation and the corresponding stimulus. To achieve this, we designed a vibrotactile device fully integrated into a prosthetic socket, capable of transmitting high-frequency contact information such as surface roughness and initial contact from interactions with external objects. The device features two compact planar vibrotactile actuators that are in direct contact with the user’s skin, providing tactile cues. These cues correspond to the acceleration profiles recorded by two IMUs placed on the distal phalanx of a soft under-actuated robotic prosthesis (SoftHand Pro). We characterised the system from a psychophysical perspective with fifteen able-bodied participants, measuring their Just Noticeable Difference (JND) in discriminating vibrotactile cues delivered to the index finger during the exploration of various sandpapers. Furthermore, we conducted a pilot experiment with one user of the SoftHand Pro prosthesis, implementing a task called Active Texture Identification to determine if our feedback could enhance the user’s ability to discriminate between different textures. The results indicate that the device effectively conveys contact and texture cues, which users can readily detect and distinguish.

This work presents the usage of a device for teleoperation tasks, specifically evaluating the impact of wearable haptics for force feedback in teleimpedance control for dual-arm robotic teleoperation. Robotic teleoperation enables humans to safely complete exploratory procedures in remote locations, such as deep-sea exploration or building assessments following natural disasters. Successful task completion in these environments requires meaningful dual-arm robotic coordination and a proper understanding of the environment, capabilities that are inherent to humans via impedance regulation and haptic interactions but challenging to achieve in telerobotic systems. Teleimpedance control has facilitated impedance regulation in such applications, while bilateral teleoperation systems aim to restore haptic sensation to the operator, often at the expense of stability or workspace size. Wearable haptic devices, such as the Clenching Upper-Limb Force Feedback (CUFF), have the potential to inform the operator of key forces during task completion while maintaining stability and transparency. In our study, able-bodied participants completed a peg-in-hole, box placement task, aiming to seat as many boxes as possible within the trial period. Experiments were conducted using both transparent and opaque boxes. With the opaque boxes, participants achieved a higher number of successful placements with haptic feedback, and the experimental trend showed higher mean interaction forces. Results suggest that the provision of wearable haptic feedback may increase operator confidence when visual cues are obscured.

This final chapter synthesizes the findings and discussions presented throughout this book, emphasizing the pivotal role of a user-centred approach in developing wearable haptic solutions. The research spans the understanding of neural plasticity and its implications in sensory substitution to the design, development, and validation of novel haptic devices. By involving users from the initial stages of design, this work has led to the creation of innovative devices such as CUFF, WISH, and VIBES, which provide force feedback, and first contact information. These devices have been rigorously tested in diverse applications, including rehabilitation, prosthesis control, and navigation for visually impaired individuals. The findings underscore the importance of user involvement in creating effective and widely accepted haptic feedback systems. This study highlights that relying solely on state-of-the-art technology is insufficient. Instead, an integrated approach that incorporates insights from neuroscience and a thorough understanding of user needs is crucial. Future developments and lessons learned from this research are also discussed, providing a pathway for continued innovation and improvement in the field.