Dense Tracking with Range Cameras Using Key Frames

We present a low cost localization system that exploits dense image information to continuously track the position of a range camera in 6DOF. This work has two main contributions: First, the localization of the camera is performed with respect to a set of keyframes selected according to a spatial criteria producing a less populated and more uniform distribution of keyframes in space. This allows us to avoid the computational overload caused by having to estimate a depthmap at the frame rate of the camera as it is common in other dense sequential methods. Second, we propose a two-stage approach to compute the current location of the camera with respect to its closest keyframe. During the first stage, our system calculates an initial relative pose estimate from a sparse set of 3D to 2D point correspondences. This estimate is then refined during the second stage using a dense image alignment. The refinement step is stated as a Non Linear Least Squares (NLQs) optimisation embedded in a coarse to fine approach that minimizes the photo-consistency error between the current image and a warped version of the image associated to the closest keyframe and its depth map.

To validate the accuracy of our system, we conducted experiments using datasets with perfectly known trajectory and with both, perfect ray-traced images and images with noise and blur. We also evaluate the accuracy of the system using datasets with RGBD images taken at different frame-rates, and the improvements in convergence due to our coarse-to-find approach. Our assessment shows that our system is able to achieve millimeter accuracy. Most of the expensive calculations are carried out by exploiting parallel computation on a GPU.