Geometric primitive refinement for structured light cameras

Three-dimensional camera systems are useful sensors for several higher level vision tasks like navigation, environment mapping or dimensioning. However, the raw 3-D data is for many algorithms not the best representation. Instead, many methods rely on a more abstract scene description, where the scene is represented as a collection of geometric primitives like planes, spheres, or even more complex models. These primitives are commonly estimated on individual point measurements, which are directly affected by the measurement errors of the sensor. This paper proposes a method for refining the parameters of geometric primitives for structured light cameras with spatially varying patterns. In contrast to fitting the model to a set of 3-D point measurements, we propose to use all information that belongs to a particular object simultaneously to directly fit the model to the image, without the detour of calculating disparities. To this end, we propose a novel calibration procedure which recovers the unknown internal parameters of the range sensors and reconstructs the unknown projected pattern. This is particularly necessary for consumer-structured light sensors whose internals are not available to the user. After calibration, a coarse model fit is considerably refined by comparing the observed structured light dot pattern with a predicted virtual view of the projected virtual pattern. The calibration and the refinement methods are evaluated on three geometric primitives: planes, spheres, and cuboids. The orientations of the plane normals are improved by more than 60%, and plane distances by more than 30% compared to the baseline. Furthermore, the initial parameters of spheres and cuboids are refined by more than 50 and 30%. The method also operates robustly on highly textured plane segments, and at ranges that have not been considered during calibration.