Panoptic Feature Pyramid Networks

@article{Kirillov2019PanopticFP,
  title={Panoptic Feature Pyramid Networks},
  author={Alexander Kirillov and Ross B. Girshick and Kaiming He and Piotr Doll{\'a}r},
  journal={2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019},
  pages={6392-6401}
}
The recently introduced panoptic segmentation task has renewed our community's interest in unifying the tasks of instance segmentation (for thing classes) and semantic segmentation (for stuff classes. [] Key Method Our approach is to endow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone.
[PDF] Semantic Reader
681 Citations
Highly Influential Citations
176
Background Citations
210
Methods Citations
394
Results Citations
17

Figures and Tables from this paper

681 Citations

Multi-task Network for Panoptic Segmentation in Automated Driving

This approach solves all three tasks including panoptic segmentation with an end-to-end learnable fully convolutional neural network, and builds upon the Mask R-CNN framework with a shared backbone and individual network heads for each task.

Panoptic Segmentation

A novel panoptic quality (PQ) metric is proposed that captures performance for all classes (stuff and things) in an interpretable and unified manner and is performed a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task.

Fast Panoptic Segmentation with Soft Attention Embeddings

A novel, fast and accurate single-stage panoptic segmentation network that employs a shared feature extraction backbone and three network heads for object detection, semantic segmentation, instance-level attention masks is introduced.

Learning panoptic segmentation through feature discriminability

LPSNet: A lightweight solution for fast panoptic segmentation

This work proposes an one-stage approach called Lightweight Panoptic Segmentation Network (LPSNet), which does not involve a proposal, anchor or mask head, and designs a parameter-free head to merge the per-pixel bounding box and semantic prediction into panoptic segmentation output.

SSPSNet: a single shot panoptic segmentation network for accurate scene parsing

This paper designs a parameter-free identical mapping connection module that increases shortcut on the mask segmentation, FCOS classification and regression branches, respectively, to extract more expressive feature maps for instance segmentation and object detection subtasks.

EPSNet: Efficient Panoptic Segmentation Network with Cross-layer Attention Fusion

This work proposes an Efficient Panoptic Segmentation Network (EPSNet) to tackle the panoptic segmentation tasks with fast inference speed and adopts a module called "cross-layer attention fusion module", which aggregates the multi-scale features with attention mechanism helping them capture the long-range dependencies between each other.

PRN: Panoptic Refinement Network

The Panoptic Refinement Network (PRN) is proposed, which takes masks from base panoptic segmentation models and refines them jointly to produce coherent results, and extends the offset map-based architecture ofPanoptic-Deeplab with several novel ideas.

Attention-Guided Unified Network for Panoptic Segmentation

The underlying relationship between FG objects and BG contents is revealed, in particular, FG objects provide complementary cues to assist BG understanding, and the Attention-guided Unified Network (AUNet) is named, a unified framework with two branches for FG and BG segmentation simultaneously.

CFNet: Learning Correlation Functions for One-Stage Panoptic Segmentation

This work proposes to first predict semantic-level and instance-level correlations among different locations that are utilized to enhance the backbone features, and then feed the improved discriminative features into the corresponding segmentation heads, respectively.
...

References

SHOWING 1-10 OF 63 REFERENCES

Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network

This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics and is faster and more memory efficient than training and predicting with two networks, as done in previous work.

Panoptic Segmentation

A novel panoptic quality (PQ) metric is proposed that captures performance for all classes (stuff and things) in an interpretable and unified manner and is performed a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task.

Weakly- and Semi-Supervised Panoptic Segmentation

A weakly supervised model that jointly performs both semantic- and instance-segmentation and is able to segment both “thing” and “stuff” classes, and thus explain all the pixels in the image.

InstanceCut: From Edges to Instances with MultiCut

To reason globally about the optimal partitioning of an image into instances, the authors combine these two modalities into a novel MultiCut formulation, which achieves the best result among all published methods, and performs particularly well for rare object classes.

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%.

Rethinking Atrous Convolution for Semantic Image Segmentation

The proposed `DeepLabv3' system significantly improves over the previous DeepLab versions without DenseCRF post-processing and attains comparable performance with other state-of-art models on the PASCAL VOC 2012 semantic image segmentation benchmark.

Triply Supervised Decoder Networks for Joint Detection and Segmentation

A framework called TripleNet is proposed to deeply boost joint object detection and semantic segmentation and outperforms all the other one-stage methods on both two tasks by a single network.

DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs

This work addresses the task of semantic image segmentation with Deep Learning and proposes atrous spatial pyramid pooling (ASPP), which is proposed to robustly segment objects at multiple scales, and improves the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models.

Learning to Refine Object Segments

This work proposes to augment feedforward nets for object segmentation with a novel top-down refinement approach that is capable of efficiently generating high-fidelity object masks and is 50 % faster than the original DeepMask network.

Fully convolutional networks for semantic segmentation

The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.
...