Instance and Panoptic Segmentation Using Conditional Convolutions

  title={Instance and Panoptic Segmentation Using Conditional Convolutions},
  author={Zhi Tian and Bowen Zhang and Hao Chen and Chunhua Shen},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
We propose a simple yet effective framework for instance and panoptic segmentation, termed CondInst (conditional convolutions for instance and panoptic segmentation). In the literature, top-performing instance segmentation methods typically follow the paradigm of Mask R-CNN and rely on ROI operations (typically ROIAlign) to attend to each instance. In contrast, we propose to attend to the instances with dynamic conditional convolutions. Instead of using instance-wise ROIs as inputs to the… 

SE-PSNet: Silhouette-based Enhancement Feature for Panoptic Segmentation Network

Box2Mask: Box-supervised Instance Segmentation via Level-set Evolution

This paper presents a novel single-shot instance segmentation approach, namely Box2Mask, which integrates the classical level-set evolution model into deep neural network learning to achieve accurate mask prediction with only bounding box supervision.

Instance Motion Tendency Learning for Video Panoptic Segmentation

A simple yet effective Instance Motion Tendency Network (IMTNet) for video panoptic segmentation that learns a global motion tendency map for instance association, and a hierarchical classifier for motion boundary refinement.

QueryInst: Parallelly Supervised Mask Query for Instance Segmentation

This paper presents QueryInst, a query based instance segmentation method driven by parallel supervision on dynamic mask heads that achieves the best performance among all online VIS approaches and strikes a decent speed-accuracy trade-off.

PPMN: Pixel-Phrase Matching Network for One-Stage Panoptic Narrative Grounding

A one-stage end-to-end Pixel-Phrase Matching Network (PPMN), which directly matches each phrase to its corresponding pixels instead of region proposals and outputs panoptic segmentation by simple combination, which achieves new state-of-the-art performance on the PNG benchmark with 4.0 absolute Average Recall gains.

Panoptic Segmentation: A Review

This is the first comprehensive review of existing panoptic segmentation methods to the best of the authors’ knowledge and provides a comparison of the performance of existing solutions to inform the state-of-the-art and identify their limitations and strengths.

UniInst: Unique Representation for End-to-End Instance Segmentation

  • Yimin OuRui Yang Xiu Li
  • Computer Science
  • 2022
This work designs an instance-aware one-to-one assignment scheme, namely OYOR, which dynamically assigns one unique representation to each instance according to the matching quality between predictions and ground truths, and achieves competitive performance against mainstream methods on COCO test-dev.

PyrAug: Gaussian Pyramid-based Multi-scale Data Augmentation for Panoptic Plant Segmentation with Ambiguous Boundary

A Gaussian pyramid - based multi -scale data augmentation for panoptic segmentation of plants with ambiguous boundaries, named PyrAug, which attempt to hierarchically enhance images at different levels, in which the network can effectively learn different feature distributions among plants and thus predict the respective masks.

Instances as Queries

For the first time, it is demonstrated that a simple end-to-end query based framework can achieve the state-of-the-art performance in various instance-level recognition tasks.

Polar Ray: A Single-stage Angle-free Detector for Oriented Object Detection in Aerial Images

A new anchor-free oriented object detector, Polar Ray Network (PRNet), where object keypoints are represented by polar coordinates without angle regression, and a dynamic PointConv module to optimize the regression of polar ray by incorporating object corner features is introduced.



Fully Convolutional Networks for Panoptic Segmentation

This approach aims to represent and predict foreground things and background stuff in a unified fully convolutional pipeline and outperforms previous box-based and -free models with high efficiency on COCO, Cityscapes, and Mapillary Vistas datasets with single scale input.

UPSNet: A Unified Panoptic Segmentation Network

A parameter-free panoptic head is introduced which solves thepanoptic segmentation via pixel-wise classification and first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolving the conflicts between semantic and instance segmentation.

Panoptic Feature Pyramid Networks

This work endsow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone, and shows it is a robust and accurate baseline for both tasks.

BlendMask: Top-Down Meets Bottom-Up for Instance Segmentation

The proposed BlendMask can effectively predict dense per-pixel position-sensitive instance features with very few channels, and learn attention maps for each instance with merely one convolution layer, thus being fast in inference.

SOLOv2: Dynamic and Fast Instance Segmentation

State-of-the-art results in object detection (from the authors' mask byproduct) and panoptic segmentation show the potential to serve as a new strong baseline for many instance-level recognition tasks besides instance segmentation.

SOLO: Segmenting Objects by Locations

A new, embarrassingly simple approach to instance segmentation in images by introducing the notion of "instance categories", which assigns categories to each pixel within an instance according to the instance's location and size thus nicely converting instance mask segmentation into a classification-solvable problem.

Instance-Sensitive Fully Convolutional Networks

This paper develops FCNs that are capable of proposing instance-level segment candidates that do not have any high-dimensional layer related to the mask resolution, but instead exploits image local coherence for estimating instances.

Semi-convolutional Operators for Instance Segmentation

It is shown theoretically and empirically that constructing dense pixel embeddings that can separate object instances cannot be easily achieved using convolutional operators, and that simple modifications, which are called semi-convolutional, have a much better chance of succeeding at this task.

PolarMask: Single Shot Instance Segmentation With Polar Representation

  • Enze XiePei Sun Ping Luo
  • Computer Science
    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
In this paper, we introduce an anchor-box free and single shot instance segmentation method, which is conceptually simple, fully convolutional and can be used by easily embedding it into most

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

This work addresses the task of semantic image segmentation with Deep Learning and proposes atrous spatial pyramid pooling (ASPP), which is proposed to robustly segment objects at multiple scales, and improves the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models.