Developing Spatial Visualization Skills with Virtual Reality and Hand Tracking

Spatial visualization skills are the cognitive skills required to mentally comprehend and manipulate 2D and 3D shapes. Spatial visualization skills are recognized as a crucial part of STEM education. Moreover, these skills have been linked to both students’ capacity for self-monitored learning and GPA in STEM programs. Also, many STEM fields have reported a correlation between spatial visualization skills and the level of academic success of students. Unfortunately, many students have significantly underdeveloped spatial visualization skills. Students traditionally learn spatial visualization skills via the manipulation of 3D objects and drawings. Because Virtual Reality (VR) facilitates the manipulation of 3D virtual objects, it is an effective medium to teach spatial visualization skills. In addition, it has been found that students are more motivated to learn and perform better when taught in an immersive VR environment. Several studies have presented promising results on leveraging VR to teach spatial visualization skills. However, in most of these studies, the users are not able to naturally interact with the 3D virtual objects. Additionally, many of the current VR applications fail to implement spatial presence for the user, which would lead to more effective instruction. The level of immersivity a VR application offers can have a direct impact on the users’ “first-person” experience and engagement. In light of this, a VR application integrated with hand tracking technology designed to develop spatial visualization skills of STEM students is presented. The inclusion of a user’s hand in the virtual environment could increase the users’ sense of spatial presence and first-person experience. In addition, this could act as an intuitive input method for beginners. The objective of this work is to introduce this VR application as well as future work on how to leverage Reinforcement Learning to automatically generate new 3D virtual objects [github.​com/​lopezbec/​VR_​SpatialVisualiza​tionApp].