Rapid Reconstruction and Simulation of Real Characters in Mixed Reality Environments

This paper presents a comparison of latest software and hardware methods for rapid reconstruction of real humans using as an input RGB or RGB-D images, and base on this comparison is introduced the pipeline that produces high realistic reconstructions in a reasonable amount of time, suitable for real-time Virtual Reality (VR), Augmented Reality (AR) as well as Holographic Mixed Reality (HMR). In this work, we also present and compare usage of latest VR and AR Head Mounted displays (HDMs), which are Microsoft Hololens, Oculus Rift and mobile AR. Specifically, we compare the immersion experience, interaction system, field of view and level of presence that each of these technologies provide. We demonstrate our results at Asinou church, a UNESCO Cultural Heritage monument located in Cyprus. Our reconstructed virtual narrator is the real priest of Asinou church which gives a virtual tour in the church. This interactive virtual narrator supports a range of different capabilities like performing gestures, speech and lip synchronization.