• Corpus ID: 238583816

Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach

  title={Combinatorial Optimization for Panoptic Segmentation: A Fully Differentiable Approach},
  author={Ahmed Abbas and Paul Swoboda},
We propose a fully differentiable architecture for simultaneous semantic and instance segmentation (a.k.a. panoptic segmentation) consisting of a convolutional neural network and an asymmetric multiway cut problem solver. The latter solves a combinatorial optimization problem that elegantly incorporates semantic and boundary predictions to produce a panoptic labeling. Our formulation allows to directly maximize a smooth surrogate of the panoptic quality metric by backpropagating the gradient… 

SUNet: Scale-aware Unified Network for Panoptic Segmentation

An end-to-end Scale-aware Unified Network (SUNet), which is more adaptable to multi-scale objects and based on a query-independent formulation and brings small parameter increments.

Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation

This paper presents a novel framework to integrate both semantic and instance contexts for panoptic segmentation, and proposes a novel Panoptic, Instance, and Semantic Relations (PISR) module to exploit such contexts.

RAMA: A Rapid Multicut Algorithm on GPU

  • A. AbbasP. Swoboda
  • Computer Science
    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2022
This work proposes a highly parallel primal-dual algorithm for the multicut problem, a classical graph clustering problem widely used in machine learning and computer vision, and shows resulting one to two orders-of-magnitudes improvements in execution speed without sac-rificing solution quality compared to traditional sequential algorithms that run on CPUs.



UPSNet: A Unified Panoptic Segmentation Network

A parameter-free panoptic head is introduced which solves thepanoptic segmentation via pixel-wise classification and first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolving the conflicts between semantic and instance segmentation.

Seamless Scene Segmentation

This work introduces a novel, CNN-based architecture that can be trained end-to-end to deliver seamless scene segmentation results by means of a panoptic output format, going beyond the simple combination of independently trained segmentation and detection models.

Panoptic Feature Pyramid Networks

This work endsow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone, and shows it is a robust and accurate baseline for both tasks.

SOGNet: Scene Overlap Graph Network for Panoptic Segmentation

This study aims to model overlap relations among instances and resolve them for panoptic segmentation, Inspired by scene graph representation, and forms the overlapping problem as a simplified case, named scene overlap graph.

The Semantic Mutex Watershed for Efficient Bottom-Up Semantic Instance Segmentation

A greedy algorithm for joint graph partitioning and labeling derived from the efficient Mutex Watershed partitioning algorithm that optimizes an objective function closely related to the Symmetric Multiway Cut objective and empirically shows efficient scaling behavior.

EfficientPS: Efficient Panoptic Segmentation

The Efficient Panoptic Segmentation (EfficientPS) architecture is introduced that consists of a shared backbone which efficiently encodes and fuses semantically rich multi-scale features and incorporates a new semantic head that aggregates fine and contextual features coherently.

Panoptic Segmentation

A novel panoptic quality (PQ) metric is proposed that captures performance for all classes (stuff and things) in an interpretable and unified manner and is performed a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task.

MaX-DeepLab: End-to-End Panoptic Segmentation with Mask Transformers

MaX-DeepLab, the first end-to-end model for panoptic segmentation, is presented, and shows a significant 7.1% PQ gain in the box-free regime on the challenging COCO dataset, closing the gap between box-based and box- free methods for the first time.

InstanceCut: From Edges to Instances with MultiCut

To reason globally about the optimal partitioning of an image into instances, the authors combine these two modalities into a novel MultiCut formulation, which achieves the best result among all published methods, and performs particularly well for rare object classes.

Conditional Convolutions for Instance Segmentation

A simpler instance segmentation method that can achieve improved performance in both accuracy and inference speed on the COCO dataset, and outperform a few recent methods including well-tuned Mask RCNN baselines, without longer training schedules needed.