Introduction

Much early machine-vision research in the modern signals-to-symbols paradigm was concerned with the interpretation of scenes from the “blocks world.” Line drawings of simple geometric objects were analyzed to infer the shapes of individual objects. More recent research has focused on the recognition of man-made objects, such as industrial parts in a factory setting, roads in an aerial photograph, and furniture in an office environment. In these systems, several complicating factors that were not present in the blocks world had to be addressed: namely noisy images, imperfect geometric models, and complex lighting. The complexity of description necessary for recognition was greater than that required for the blocks world. A logical next step in this progression is the interpretation of ground-level images of natural outdoor scenes. In the manufactured world, three-dimensional (3D) edges and surfaces are an adequate intermediate representation, but for the natural world, such shape descriptions are insufficient and perhaps inappropriate. By designing a vision system for interpreting ground-level scenes of the outdoor world, we hope to provide a new basis for a theory of computational image understanding in complex domains.