ABSTRACT
Virtual and Augmented Reality (VR and AR) are two fast growing mediums, not only in the entertainment industry but also in health, education and engineering. A good VR or AR application seamlessly merges the real and virtual world, making the user feels fully immersed. Traditionally, a computer-generated object can be interacted with using controllers or hand gestures [HTC 2019; Microsoft 2019; Oculus 2019]. However, these motions can feel unnatural and do not accurately represent the motion of interacting with a real object. On the other hand, a physical object can be used to control the motion of a virtual object. At present, this can be done by tracking purely rigid motion using an external sensor [HTC 2019]. Alternatively, a sparse number of markers can be tracked, for example using a motion capture system, and the positions of these used to drive the motion of an underlying non-rigid model. However, this approach is sensitive to changes in marker position and occlusions and often involves costly non-standard hardware [Vicon 2019]. In addition, these approaches often require a virtual model to be manually sculpted and rigged which can be a time consuming process. Neural networks have been shown to be successful tools in computer vision, with several key methods using networks for tracking rigid and non-rigid motion in RGB images [Andrychowicz et al. 2018; Kanazawa et al. 2018; Pumarola et al. 2018]. While these methods show potential, they are limited to using multiple RGB cameras or large, costly amounts of labelled training data.
Supplemental Material
- M. Andrychowicz, B. Baker, M. Chociej, R. Józefowicz, B. McGrew, J. W. Pachocki, A. Petron, M. Plappert, G. Powell, A. Ray, J. Schneider, S. Sidor, J. Tobin, P. Welinder, L. Weng, and W. Zaremba. 2018. Learning Dexterous In-Hand Manipulation. CoRR (2018).Google Scholar
- R. D. Cook. 1989. Concepts and applications of finite element analysis. (3rd ed.).Google Scholar
- K. He, X. Zhang, S. Ren, and J. Sun. 2016. Identity mappings in deep residual networks. In European conference on computer vision. 630--645.Google Scholar
- HTC. 2019. Discover Virtual Reality Beyond Imagination. https://www.vive.com/uk/.Google Scholar
- A. Kanazawa, M. J Black, D. W Jacobs, and J. Malik. 2018. End-to-end recovery of human shape and pose. In Proceedings of the CVPR. 7122--7131.Google Scholar
- Microsoft. 2019. Microsoft HoloLens | Mixed Reality Technology for Business. https://www.microsoft.com/en-us/hololens.Google Scholar
- Oculus. 2019. Oculus Rift. https://www.oculus.com/rift/.Google Scholar
- A. Pumarola, A. Agudo, A. Porzi, L. and Sanfeliu, V. Lepetit, and F. Moreno-Noguer. 2018. Geometry-aware network for non-rigid shape prediction from a single view. In Proceedings of CVPR. 4681--4690.Google Scholar
- Vicon. 2019. Motion Capture Systems. https://www.vicon.com/.Google Scholar
- S. Zagoruyko and N. Komodakis. 2016. Wide residual networks. arXiv preprint arXiv:1605.07146 (2016).Google Scholar
Index Terms
- VRProp-net: real-time interaction with virtual props
Recommendations
Augmented reality on cloth with realistic illumination
Augmented reality (AR) is the concept of inserting virtual objects into real scenes. Often, augmentations are aligned with rigid planar objects in the scene. However, a more difficult task is to align non-rigid augmentations with flexible objects like ...
Taking steps: the influence of a walking technique on presence in virtual reality
Special issue on virtual reality software and technologyThis article presents an interactive technique for moving through an immersive virtual environment (or “virtual reality”). The technique is suitable for applications where locomotion is restricted to ground level. The technique is derived from the idea ...
Net Juggler: Running VR Juggler with Multiple Displays on a Commodity Component Cluster
VR '02: Proceedings of the IEEE Virtual Reality Conference 2002Net Juggler is an open source library that turns a commodity component cluster running the VR Juggler platform on each node into a single VR Juggler image cluster. Application parallelization is transparent to the user and leads to high performance ...
Comments