An Accurate Method for Line Detection and Manhattan Frame Estimation

We address the problem of estimating the rotation of a camera relative to the canonical frame of an urban scene, from a single image. Solutions generally rely on the so-called ‘Manhattan World’ assumption [1] that the major structures in the scene conform to three orthogonal principal directions. This can be expressed as a generative model in which the dense gradient map of the image is explained by a mixture of the three principal directions and a background process [2]. It has recently been shown that using sparse oriented edges rather than the dense gradient map leads to substantial gains in both accuracy and speed [3]. Here we explore whether further gains can be made by basing inference on even sparser extended lines. Standard Houghing techniques suffer from quantization errors and noise that make line extraction unreliable. Here we introduce a probabilistic line extraction technique that eliminates these problems through two innovations. First, we accurately propagate edge uncertainty from the image to the Hough map through a bivariate normal kernel that uses natural image statistics, resulting in a non-stationary ‘soft-voting’ technique. Second, we eliminate multiple responses to the same line by updating the Hough map dynamically as each line is extracted. We evaluate the method on a standard benchmark dataset [3], showing that the resulting line representation supports reliable estimation of the Manhattan frame, bettering the accuracy of previous edge-based methods by a factor of 2 and the gradient-based Manhattan World method by a factor of 5.