An Exploratory Comparison of the Visual Quality of Virtual Reality Systems Based on Device-Independent Testsets

Nowadays, several different devices exist to offer virtual, augmented and mixed reality to show artificial objects. Measurements of the quality or the correctness of their resulting visual structures are not developed as sophisticated as in the classical areas of 2D image and video processing. Common testsets for image and video processing frequently contain sequences from the real world to reproduce their intrinsic characteristics and properties as well as artificial structures to provoke potential visual errors (see Fig. 1a). These common but traditional testsets are nowadays faced with rapid technical developments and changes like HD, UHD etc. improved surround sound or multiple data streams. That results in a limitation of the testsets usability and their ability to evoke visual errors. To overcome those limitations, we developed a system to create device-independent testsets to be used in the area of virtual reality devices and 3D environments. We conduct an empirical evaluation of most recent virtual reality devices like HTC Vive and Zeiss Cinemizer OLED, aiming to explore whether the technical hardware properties of the devices or the provided software interfaces may introduce errors in the visual representation. The devices are going to be evaluated by a group with technical skills and mostly advanced knowledge in computer graphics. All perceived visual and technical saliences are recorded in order to evaluate the correctness and the quality of the devices and the constraints.