Virtual Reality Hand Tracking for Immersive Telepresence in Rehabilitative Serious Gaming

Intelligent systems face an increasingly complex array of challenges in the rapidly evolving landscape of human-device interactions. To address these challenges, there arises a pressing need for the development of refined methods of information transfer, capable of capturing subtle nuances such as body language and tonal subtleties. The system presented in this paper is designed to highlight a novel method of refined human-device interactions and its potential impact on the medical domain. Leveraging the Oculus SDK in the Unity Game Engine alongside the Oculus VR headset, our system allows users to interact with virtual objects naturally, mirroring real-world hand movements while simultaneously promoting upper limb rehabilitation through engaging gameplay scenarios. The integration of haptic feedback enriches the immersive experience, enabling users to not only visualize but also feel their virtual interactions. The system accommodates varying levels of mobility, adapting its complexity to individual progress. Furthermore, our VR rehabilitation system has garnered positive outcomes from user assessments, demonstrating the system’s effectiveness and user satisfaction. While limited in addressing severe upper limb impairments, the system’s flexibility allows for modular improvements and broader clinical integration.