Seeing What is Not There: Learning Context to Determine Where Objects are Missing

@article{Sun2017SeeingWI,
  title={Seeing What is Not There: Learning Context to Determine Where Objects are Missing},
  author={J. Sun and David W. Jacobs},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={1234-1242}
}
  • J. Sun, D. Jacobs
  • Published 2017
  • Computer Science
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
Most of computer vision focuses on what is in an image. We propose to train a standalone object-centric context representation to perform the opposite task: seeing what is not there. Given an image, our context model can predict where objects should exist, even when no object instances are present. Combined with object detection results, we can perform a novel vision task: finding where objects are missing in an image. Our model is based on a convolutional neural network structure. With a… Expand
Object detection can be improved using human-derived contextual expectations
TLDR
For the first time, human-derived scene expectations can be used to improve object detection performance in machines and augmented state-of-the-art object detectors with these human- derived expectations on novel scenes produce a significant improvement. Expand
Non-parametric spatial context structure learning for autonomous understanding of human environments
TLDR
The spatial context based system is tested to successfully conclude that it can capture spatio structural information to do joint object classification to not only act as a vision aide, but sometimes even perform on par with appearance based robotic vision. Expand
Deep neural networks can be improved using human-derived contextual expectations
TLDR
It is shown for the first time that human-derived scene expectations can be used to improve object detection performance in machines and augmenting deep neural networks with human- derived context features improves their performance, suggesting that humans learn scene context separately unlike deep networks. Expand
Learning a Layout Transfer Network for Context Aware Object Detection
TLDR
This work presents a context aware object detection method based on a retrieve-and-transform scene layout model that provides consistent performance improvements to the state-of-the-art object detection baselines on a variety of challenging tasks in the traffic surveillance and the autonomous driving domains. Expand
Context-aware Synthesis and Placement of Object Instances
TLDR
This paper proposes an end-to-end trainable neural network for the task of inserting an object instance mask of a specified class into the semantic label map of an image and devise a learning procedure that leverage both supervised and unsupervised data. Expand
Where and Who? Automatic Semantic-Aware Person Composition
TLDR
A novel branching Convolutional Neural Network that jointly predicts candidate person locations given a background image and uses pre-trained deep feature representations to retrieve person instances from a large segment database is developed. Expand
Machine vision benefits from human contextual expectations
TLDR
This work developed a paradigm to measure human contextual expectations and augmented decisions made by deep neural networks with human-derived contextual expectations, suggesting that contextual representations are qualitatively different in humans andDeep neural networks. Expand
Unconstrained Foreground Object Search
TLDR
A cost-free, scalable approach for creating a large-scale training dataset with a variety of foreground objects of differing semantic categories per image location and introduces a solution that supports efficient search by encoding the background image in the same latent space as the candidate foreground objects. Expand
Hidden Footprints: Learning Contextual Walkability from 3D Human Trails
TLDR
This work devise a training strategy designed for such sparse labels, combining a class-balanced classification loss with a contextual adversarial loss that demonstrates a model that learns to predict a walkability map from a single image. Expand
Detecting Nonexistent Pedestrians
TLDR
The method builds upon a combination of generative and discriminative procedures to achieve the perceptual capability of figuring out missing visual information and can capture the idea of where the reasonable places are for pedestrians to walk or stand in a street scene. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 20 REFERENCES
Tracking the invisible: Learning where the object might be
TLDR
This work proposes a method to learn supporters which are, be it only temporally, useful for determining the position of the object of interest and exploits the General Hough Transform strategy. Expand
OverFeat: Integrated Recognition, Localization and Detection using Convolutional Networks
TLDR
This integrated framework for using Convolutional Networks for classification, localization and detection is the winner of the localization task of the ImageNet Large Scale Visual Recognition Challenge 2013 and obtained very competitive results for the detection and classifications tasks. Expand
Context Encoders: Feature Learning by Inpainting
TLDR
It is found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures, and can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods. Expand
Beyond Categories: The Visual Memex Model for Reasoning About Object Relationships
TLDR
An exemplar-based model of objects and their relationships is presented, the Visual Memex, that encodes both local appearance and 2D spatial context between object instances and may be the critical missing ingredient in scene understanding systems. Expand
An empirical study of context in object detection
TLDR
This paper presents an empirical evaluation of the role of context in a contemporary, challenging object detection task - the PASCAL VOC 2008, using top-performing local appearance detectors as baseline and evaluates several different sources of context and ways to utilize it. Expand
The Role of Context for Object Detection and Semantic Segmentation in the Wild
TLDR
A novel deformable part-based model is proposed, which exploits both local context around each candidate detection as well as global context at the level of the scene, which significantly helps in detecting objects at all scales. Expand
The role of context in object recognition
In the real world, objects never occur in isolation; they co-vary with other objects and particular environments, providing a rich source of contextual associations to be exploited by the visualExpand
Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks
TLDR
This work introduces a Region Proposal Network (RPN) that shares full-image convolutional features with the detection network, thus enabling nearly cost-free region proposals and further merge RPN and Fast R-CNN into a single network by sharing their convolutionAL features. Expand
Objects in Context
TLDR
This work proposes to incorporate semantic object context as a post-processing step into any off-the-shelf object categorization model using a conditional random field (CRF) framework, which maximizes object label agreement according to contextual relevance. Expand
Visual objects in context
  • M. Bar
  • Psychology, Medicine
  • Nature Reviews Neuroscience
  • 2004
TLDR
Building on previous findings, the knowledge that is available is reviewed, specific mechanisms for the contextual facilitation of object recognition are proposed, and important open questions are highlighted. Expand
...
1
2
...