Decomposing a scene into geometric and semantically consistent regions


High-level, or holistic, scene understanding involves reasoning about objects, regions, and the 3D relationships between them. This requires a representation above the level of pixels that can be endowed with high-level attributes such as class of object/region, its orientation, and (rough 3D) location within the scene. Towards this goal, we propose a region-based model which combines appearance and scene geometry to automatically decompose a scene into semantically meaningful regions. Our model is defined in terms of a unified energy function over scene appearance and structure. We show how this energy function can be learned from data and present an efficient inference technique that makes use of multiple over-segmentations of the image to propose moves in the energy-space. We show, experimentally, that our method achieves state-of-the-art performance on the tasks of both multi-class image segmentation and geometric reasoning. Finally, by understanding region classes and geometry, we show how our model can be used as the basis for 3D reconstruction of the scene.

DOI: 10.1109/ICCV.2009.5459211

Extracted Key Phrases

Showing 1-10 of 23 references

The PASCAL Visual Object Classes Challenge 2007 (VOC2007) Results

  • M Everingham, L Van Gool, C K I Williams, J Winn, A Zisserman
  • 2007
Showing 1-10 of 345 extracted citations
Citations per Year

546 Citations

Semantic Scholar estimates that this publication has received between 462 and 649 citations based on the available data.

See our FAQ for additional information.