The Role of Context for Object Detection and Semantic Segmentation in the Wild

@article{Mottaghi2014TheRO,
  title={The Role of Context for Object Detection and Semantic Segmentation in the Wild},
  author={Roozbeh Mottaghi and Xianjie Chen and Xiaobai Liu and Nam-Gyu Cho and Seong-Whan Lee and Sanja Fidler and Raquel Urtasun and Alan Loddon Yuille},
  journal={2014 IEEE Conference on Computer Vision and Pattern Recognition},
  year={2014},
  pages={891-898}
}
In this paper we study the role of context in existing state-of-the-art detection and segmentation approaches. Towards this goal, we label every pixel of PASCAL VOC 2010 detection challenge with a semantic category. We believe this data will provide plenty of challenges to the community, as it contains 520 additional classes for semantic segmentation and object detection. Our analysis shows that nearest neighbor based approaches perform poorly on semantic segmentation of contextual classes… 

Figures and Tables from this paper

Object-Contextual Representations for Semantic Segmentation
TLDR
This paper addresses the semantic segmentation problem with a focus on the context aggregation strategy, and presents a simple yet effective approach, object-contextual representations, characterizing a pixel by exploiting the representation of the corresponding object class.
Region-Based Semantic Segmentation with End-to-End Training
TLDR
A novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class, which improves the state-of-the-art in terms of class-average accuracy with 64.0% on SIFT Flow and 49.9% on PASCAL Context.
Exploring Person Context and Local Scene Context for Object Detection
TLDR
Two ways of using context for object detection are explored that are able to capture precise spatial relationships between the context and the object of interest, and make effective use of the appearance of the contextual region.
Deep feature based contextual model for object detection
Towards Unified Object Detection and Semantic Segmentation
TLDR
Extensive experiments on the PASCAL VOC 2010 and 2012 datasets demonstrate encouraging performance of the proposed unified framework for both object detection and semantic segmentation tasks.
Context for Object Detection via Lightweight Global and Mid-level Representations
  • Mesut Erhan Unal
  • Computer Science
    2020 25th International Conference on Pattern Recognition (ICPR)
  • 2021
TLDR
This work proposes an approach for explicitly capturing context in object detection that significantly outperforms Faster R-CNN, and performs competitively with a much more costly approach that also models context.
Region-Based Semantic Segmentation with End-to-End Training
TLDR
A novel method for semantic segmentation, the task of labeling each pixel in an image with a semantic class, is proposed, which improves the state-of-the-art in terms of class-average accuracy and is particularly accurate at object boundaries.
PASCAL Boundaries: A Class-Agnostic Semantic Boundary Dataset
TLDR
A large database consisting of more than 10k images along with ground truth boundaries between 459 semantic classes including both foreground objects and different types of background is generated and called the PASCAL Boundaries dataset, which will be released to the community.
segDeepM: Exploiting segmentation and context in deep neural networks for object detection
TLDR
This paper frames the problem as inference in a Markov Random Field, in which each detection hypothesis scores object appearance as well as contextual information using Convolutional Neural Networks, and allows the hypothesis to choose and score a segment out of a large pool of accurate object segmentation proposals.
...
...

References

SHOWING 1-10 OF 48 REFERENCES
An empirical study of context in object detection
TLDR
This paper presents an empirical evaluation of the role of context in a contemporary, challenging object detection task - the PASCAL VOC 2008, using top-performing local appearance detectors as baseline and evaluates several different sources of context and ways to utilize it.
Bottom-Up Segmentation for Top-Down Detection
TLDR
A novel deformable part-based model which exploits region-based segmentation algorithms that compute candidate object regions by bottom-up clustering followed by ranking of those regions that outperform the previous state-of-the-art on VOC 2010 test by 4%.
Describing the scene as a whole: Joint object detection, scene classification and semantic segmentation
TLDR
An approach to holistic scene understanding that reasons jointly about regions, location, class and spatial extent of objects, presence of a class in the image, as well as the scene type that outperforms the state-of-the-art on the MSRC-21 benchmark, while being much faster.
Analyzing Semantic Segmentation Using Hybrid Human-Machine CRFs
TLDR
This work "plug-in" human subjects for each of the various components in a state-of-the-art conditional random field model (CRF) on the MSRC dataset to help improve semantic segmentation.
Exploiting hierarchical context on a large database of object categories
TLDR
This paper introduces a new dataset with images that contain many instances of different object categories and proposes an efficient model that captures the contextual information among more than a hundred ofobject categories and shows that the context model can be applied to scene understanding tasks that local detectors alone cannot solve.
Extracting adaptive contextual cues from unlabeled regions
TLDR
This paper proposes a contextual cue that exploits unlabeled regions in images and learns its proposed “contextual meta-objects” using any off-the-shelf object detector, which makes the proposed cue widely accessible to the community.
Using the forest to see the trees: exploiting context for visual object detection and localization
TLDR
A probabilistic framework for encoding the relationships between context and object properties is used and it is shown how an integrated system provides improved performance, viewed as a significant step toward general purpose machine vision systems.
Learning Spatial Context: Using Stuff to Find Things
TLDR
This paper clusters image regions based on their ability to serve as context for the detection of objects and shows that the things and stuff (TAS) context model produces meaningful clusters that are readily interpretable, and helps improve detection ability over state-of-the-art detectors.
A Statistical Model for General Contextual Object Recognition
TLDR
Experiments show that spatial context considerably improves the accuracy of object recognition, and an approximate EM algorithm that uses loopy belief propagation in the inference step and iterative scaling on the pseudo-likelihood approximation in the parameter update step converges to good local solutions.
Object Recognition by Sequential Figure-Ground Ranking
We present an approach to visual object-class segmentation and recognition based on a pipeline that combines multiple figure-ground hypotheses with large object spatial support, generated by
...
...