Nonparametric Scene Parsing via Label Transfer

  title={Nonparametric Scene Parsing via Label Transfer},
  author={Ce Liu and Jenny Yuen and Antonio Torralba},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
While there has been a lot of recent work on object recognition and image understanding, the focus has been on carefully establishing mathematical models for images, scenes, and objects. In this paper, we propose a novel, nonparametric approach for object recognition and scene parsing using a new technology we name label transfer. For an input image, our system first retrieves its nearest neighbors from a large database containing fully annotated images. Then, the system establishes dense… 

A new fast approach to nonparametric scene parsing

Nonparametric scene parsing with deep convolutional features and dense alignment

This paper proposes using the state-of-the-art deep convolutional features as image descriptors to improve the semantic quality of retrieved images and includes dense alignment into the Markov Random Field inference framework to transfer labels at pixel-level accuracy.

On the Influence of Superpixel Methods for Image Parsing

The evaluation on the SIFT Flow and Barcelona dataset will show that the choice of the superpixel method is crucial for the performance of image parsing, and a new method that computes a superpixel-like over-segmentation of an image is presented that compute regions based on edge-avoiding wavelets.

Enhancing scene parsing by transferring structures via efficient low-rank graph matching

A new graph matching model incorporating low-rank and Frobenius regularization is proposed, which not only guarantees an accurate solution, but also provides high optimization efficiency via an eigen-decomposition strategy.

Sample and Filter: Nonparametric Scene Parsing via Efficient Filtering

This paper proposes to sample labeled superpixels according to an image similarity score, which allows us to obtain a balanced set of samples and formulate label transfer as an efficient filtering procedure, which lets us exploit more labeled samples than existing techniques.

Towards large-scale nonparametric scene parsing of images and video

Two novel binary encoding algorithms for large-scale approximate nearest neighbor search are presented: the bank of random rotations is data independent and does not require training, while the supervised sparse projections algorithm targets efficient search of high-dimensional labelled data.

Adaptive Nonparametric Image Parsing

This paper proposes a novel adaptive nonparametric approach that determines the sample-specific k for each test image, adaptively set to be the number of the fewest nearest superpixels that the images in the retrieval set can use to get the best category prediction.

MC-SSM: Nonparametric Semantic Image Segmentation With the ICM Algorithm

A new geometric retrieval strategy to select nearest neighbors from a database containing fully segmented and annotated images and a novel and simple energy-minimization model, which combines efficiently different global nonparametric semantic likelihood energy terms.

Superpixel Correspondence for Non-parametric Scene Parsing of Natural Images

This paper proposes a novel scalable non-parametric scene parsing system based on superpixels correspondence that achieves promising results on the LabelMe Outdoor dataset.



Nonparametric scene parsing: Label transfer via dense scene alignment

Compared to existing object recognition approaches that require training for each object category, the proposed nonparametric scene parsing system is easy to implement, has few parameters, and embeds contextual information naturally in the retrieval/alignment procedure.

SIFT Flow: Dense Correspondence across Scenes and Its Applications

SIFT flow is proposed, a method to align an image to its nearest neighbors in a large image corpus containing a variety of scenes, where image information is transferred from the nearest neighbors to a query image according to the dense scene correspondence.

SIFT Flow: Dense Correspondence across Different Scenes

A method to align an image to its neighbors in a large image collection consisting of a variety of scenes, and applies the SIFT flow algorithm to two applications: motion field prediction from a single static image and motion synthesis via transfer of moving objects.

Superparsing - Scalable Nonparametric Image Parsing with Superpixels

This paper presents a simple and effective nonparametric approach to the problem of image parsing, or labeling image regions (in this case, superpixels produced by bottom-up segmentation) with their categories, and establishes a new benchmark for the problem.

TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context

A new approach for learning a discriminative model of object classes, incorporating texture, layout, and context information efficiently, which gives competitive and visually pleasing results for objects that are highly textured, highly structured, and even articulated.

Object Recognition by Scene Alignment

This work builds a probabilistic model to transfer the labels from the retrieval set to the input image, and demonstrates the effectiveness of this approach and study algorithm component contributions using held-out test sets from the LabelMe database.

Discriminative Models for Multi-Class Object Layout

A unified model for multi-class object recognition is introduced that casts the problem as a structured prediction task and how to formulate learning as a convex optimization problem is shown.

LabelMe: A Database and Web-Based Tool for Image Annotation

A web-based tool that allows easy image annotation and instant sharing of such annotations is developed and a large dataset that spans many object categories, often containing multiple instances over a wide variety of images is collected.

Exploiting hierarchical context on a large database of object categories

This paper introduces a new dataset with images that contain many instances of different object categories and proposes an efficient model that captures the contextual information among more than a hundred ofobject categories and shows that the context model can be applied to scene understanding tasks that local detectors alone cannot solve.

Describing Visual Scenes using Transformed Dirichlet Processes

This work develops a hierarchical probabilistic model for the spatial structure of visual scenes based on the transformed Dirichlet process, a novel extension of the hierarchical DP in which a set of stochastically transformed mixture components are shared between multiple groups of data.