Look Behind You! – Using a Face Camera for Mobile Augmented Reality Odometry

Augmented reality applications provide new ways of presenting cultural heritage assets thanks to the recent advancement in the field of smart devices. Unfortunately, the construction of the hardware and lower computational power of mobile processors limit the potential of these applications. Namely, almost all current visual-inertial odometry libraries employed in smartphones require the real tracked objects to be close and contain distinguishable features, which is an issue when observing large virtual structures outdoors like historical buildings or objects on plain walls of halls or museums. This paper exploits the possibility of using the face cameras available in mobile devices for augmented reality tracking. It designs a prototype composed of iPhone and iPad devices and evaluates its contribution in two scenarios that current systems cannot handle. The results reveal the clear benefit of this approach for the cultural heritage, allowing it to operate in situations when users look up in the sky to see the roof of virtual buildings, or when they move closer to a white wall to perceive details of a virtual painting. In the end, the paper discusses the system’s limitations and proposes solutions to them.