ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors

  title={ShapeMask: Learning to Segment Novel Objects by Refining Shape Priors},
  author={Weicheng Kuo and Anelia Angelova and Jitendra Malik and Tsung-Yi Lin},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
Instance segmentation aims to detect and segment individual objects in a scene. [] Key Method ShapeMask starts with a bounding box detection and gradually refines it by first estimating the shape of the detected object through a collection of shape priors. Next, ShapeMask refines the coarse shape into an instance level mask by learning instance embeddings. The shape priors provide a strong cue for object-like prediction, and the instance embeddings model the instance specific appearance information…

Prior to Segment: Foreground Cues for Novel Objects in Partially Supervised Instance Segmentation

This work modifications a Mask R-CNN by introducing an object mask prior (OMP) for the mask head to improve mask predictions with limited labels and obtains competitive performance with the state-of-the-art, while offering a much simpler architecture.

Learning Universal Shape Dictionary for Realtime Instance Segmentation

The proposed USD-Seg adopts a linear model, sparse coding with dictionary, for object shapes, which learns a dictionary from a large collection of shape datasets, making any shape being able to be decomposed into a linear combination through the dictionary.

Learning Saliency Propagation for Semi-Supervised Instance Segmentation

This work proposes ShapeProp, which learns to activate the salient regions within the object detection and propagate the areas to the whole instance through an iterative learnable message passing module, which establishes new states of the art for semi-supervised instance segmentation.

Commonality-Parsing Network across Shape and Appearance for Partially Supervised Instance Segmentation

This work proposes to learn the underlying class-agnostic commonalities that can be generalized from mask-annotated categories to novel categories, and significantly outperforms the state-of-the-art methods on both partially supervised setting and few-shot setting for instance segmentation on COCO dataset.

One-Shot Shape-Based Amodal-to-Modal Instance Segmentation

OSSIS is proposed, a single-stage One-Shot Shape-based Instance Segmentation algorithm that produces the target object modal segmentation mask in a depth image of a scene based only on a binary shape mask of thetarget object.

ContrastMask: Contrastive Learning to Segment Every Thing

This paper proposes a new method, named ContrastMask, which learns a mask segmentation model on both base and novel categories under a unified pixel-level contrastive learning framework, and demonstrates the superiority of this method, which outperforms previous state-of-the-arts methods.

The surprising impact of mask-head architecture on novel class segmentation

The ability of mask-heads to generalize to unseen classes the strong mask generalization effect is called and it is shown that without any specialty modules or losses, the effect is general, holding across underlying detection methodologies and across different backbone networks.

Unsupervised Discovery of the Long-Tail in Instance Segmentation Using Hierarchical Self-Supervision

A method that can perform unsupervised discovery of long-tail categories in instance segmentation, through learning instance embeddings of masked regions, and achieves competitive quantitative results on LVIS as compared to the supervised and partially supervised methods.

Mask2CAD: 3D Shape Prediction by Learning to Segment and Retrieve

Mask2CAD is presented, which jointly detects objects in real-world images and for each detected object, optimizes for the most similar CAD model and its pose, and constructs a joint embedding space between the detected regions of an image corresponding to an object and 3D CAD models, enabling retrieval of CAD models for an input RGB image.

Prior to Segment: Foreground Cues for Weakly Annotated Classes in Partially Supervised Instance Segmentation

This work introduces an object mask prior (OMP) that provides the mask head with the general concept of foreground implicitly learned by the box classification head under the supervision of all classes and improves generalization to the weakly annotated classes.



Shape Sharing for Object Segmentation

A non-parametric prior is developed that transfers object shapes from an exemplar database to a test image based on local shape matching and is enforced in a graph-cut formulation to produce a pool of object segment hypotheses.

An Exemplar-Based CRF for Multi-instance Object Segmentation

A novel CRF framework is designed that jointly models object appearance, shape deformation, and object occlusion and an alternating procedure that interleaves object segmentation and shape/appearance adaptation is derived.

Learning to Segment Object Candidates

A new way to generate object proposals is proposed, introducing an approach based on a discriminative convolutional network that obtains substantially higher object recall using fewer proposals and is able to generalize to unseen categories it has not seen during training.

SCALPEL: Segmentation Cascades with Localized Priors and Efficient Learning

SCALPEL is a flexible method for object segmentation that integrates rich region-merging cues with mid- and high-level information about object layout, class, and scale into the segmentation process and uses a cascade of bottom-up segmentation models that is capable of learning to ignore boundaries early on, yet use them as a stopping criterion once the object has been mostly segmented.

Mask R-CNN

This work presents a conceptually simple, flexible, and general framework for object instance segmentation, which extends Faster R-CNN by adding a branch for predicting an object mask in parallel with the existing branch for bounding box recognition.

MaskLab: Instance Segmentation by Refining Object Detection with Semantic and Direction Features

This work presents a model, called MaskLab, which produces three outputs: box detection, semantic segmentation, and direction prediction, which is evaluated on the COCO instance segmentation benchmark and shows comparable performance with other state-of-art models.

Simultaneous Detection and Segmentation

This work builds on recent work that uses convolutional neural networks to classify category-independent region proposals (R-CNN), introducing a novel architecture tailored for SDS, and uses category-specific, top-down figure-ground predictions to refine the bottom-up proposals.

Learning to Segment Every Thing

A new partially supervised training paradigm is proposed, together with a novel weight transfer function, that enables training instance segmentation models on a large set of categories all of which have box annotations, but only a small fraction ofWhich have mask annotations.

Weakly Supervised Instance Segmentation Using Class Peak Response

This paper addresses the challenging image-level supervised instance segmentation task by exploiting class peak responses to enable a classification network for instance mask extraction, and reports state-of-the-art results on popular benchmarks, including PASCAL VOC 2012 and MS COCO.

Semantic Instance Segmentation with a Discriminative Loss Function

This work proposes an approach of combining an off-the-shelf network with a principled loss function inspired by a metric learning objective that encourages a convolutional network to produce a representation of the image that can easily be clustered into instances with a simple post-processing step.