• Corpus ID: 226221832

Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation

@article{Wu2020AutoPanopticCM,
  title={Auto-Panoptic: Cooperative Multi-Component Architecture Search for Panoptic Segmentation},
  author={Yangxin Wu and Gengwei Zhang and Hang Xu and Xiaodan Liang and Liang Lin},
  journal={ArXiv},
  year={2020},
  volume={abs/2010.16119}
}
Panoptic segmentation is posed as a new popular test-bed for the state-of-the-art holistic scene understanding methods with the requirement of simultaneously segmenting both foreground things and background stuff. The state-of-the-art panoptic segmentation network exhibits high structural complexity in different network components, i.e. backbone, proposal-based foreground branch, segmentation-based background branch, and feature fusion module across branches, which heavily relies on expert… 

Figures and Tables from this paper

Panoptic-PartFormer: Learning a Unified Model for Panoptic Part Segmentation
TLDR
This work is the best of its knowledge to solve the PPS problem via a unified and end-to-end transformer and achieves the new state-of-the-art results on both Cityscapes PPS and Pascal Context PPS datasets with at least 70% GFlops and 50% parameters decrease.
Panoptic SegFormer: Delving Deeper into Panoptic Segmentation with Transformers
TLDR
Panoptic SegFormer, a general framework for panoptic segmentation with transformers, contains three innovative components: an efficient deeply-supervised mask decoder, a query decoupling strategy, and an improved post-processing method.
Ada-Segment: Automated Multi-loss Adaptation for Panoptic Segmentation
TLDR
The extensive ablation studies reveal the ever-changing dynamics throughout the training process, necessitating the incorporation of an automated and adaptive learning strategy as presented in this paper.
PolyphonicFormer: Unified Query Learning for Depth-aware Video Panoptic Segmentation
TLDR
PolyphonicFormer, a vision transformer to unify these sub-tasks under the DVPS task and lead to more robust results, achieves the state-of-the-art results on two DVPS datasets, and ranks 1st on the ICCV-2021 BMTT Challenge video + depth track.
Panoptic Segmentation: A Review
TLDR
This is the first comprehensive review of existing panoptic segmentation methods to the best of the authors’ knowledge and provides a comparison of the performance of existing solutions to inform the state-of-the-art and identify their limitations and strengths.
Panoptic, Instance and Semantic Relations: A Relational Context Encoder to Enhance Panoptic Segmentation
TLDR
This paper presents a novel framework to integrate both semantic and instance contexts for panoptic segmentation, and proposes a novel Panoptic, Instance, and Semantic Relations (PISR) module to exploit such contexts.
Video K-Net: A Simple, Strong, and Unified Baseline for Video Segmentation
TLDR
Video K-Net is presented, a simple, strong, and unified framework for fully end-to-end video panoptic segmentation that achieves state-of-the-art video Panoptic segmentsation results on Citscapes-VPS and KITTI-STEP without bells and whistles.
DCNAS: Densely Connected Neural Architecture Search for Semantic Image Segmentation
TLDR
This work proposes a novel Densely Connected NAS framework, which directly searches the optimal network structures for the multi-scale representations of visual information, over a large-scale target dataset without proxy, and introduces a densely connected search space to cover an abundance of mainstream network designs.
NASOA: Towards Faster Task-oriented Online Fine-tuning with a Zoo of Models
TLDR
A joint Neural Architecture Search and Online Adaption framework named NASOA is proposed towards a faster task-oriented fine-tuning upon the request of users and proposes a novel joint block and macro level search space to enable a flexible and effi-cient search.
...
...

References

SHOWING 1-10 OF 46 REFERENCES
Attention-Guided Unified Network for Panoptic Segmentation
TLDR
The underlying relationship between FG objects and BG contents is revealed, in particular, FG objects provide complementary cues to assist BG understanding, and the Attention-guided Unified Network (AUNet) is named, a unified framework with two branches for FG and BG segmentation simultaneously.
UPSNet: A Unified Panoptic Segmentation Network
TLDR
A parameter-free panoptic head is introduced which solves thepanoptic segmentation via pixel-wise classification and first leverages the logits from the previous two heads and then innovatively expands the representation for enabling prediction of an extra unknown class which helps better resolving the conflicts between semantic and instance segmentation.
Panoptic Feature Pyramid Networks
TLDR
This work endsow Mask R-CNN, a popular instance segmentation method, with a semantic segmentation branch using a shared Feature Pyramid Network (FPN) backbone, and shows it is a robust and accurate baseline for both tasks.
BANet: Bidirectional Aggregation Network With Occlusion Handling for Panoptic Segmentation
TLDR
A novel deep panoptic segmentation scheme based on a bidirectional learning pipeline is proposed and a plug-and-play occlusion handling algorithm is introduced to deal with the occlusions between different object instances.
Panoptic Segmentation
TLDR
A novel panoptic quality (PQ) metric is proposed that captures performance for all classes (stuff and things) in an interpretable and unified manner and is performed a rigorous study of both human and machine performance for PS on three existing datasets, revealing interesting insights about the task.
Bidirectional Graph Reasoning Network for Panoptic Segmentation
TLDR
A Bidirectional Graph Reasoning Network (BGRNet), which incorporates graph structure into the conventional panoptic segmentation network to mine the intra-modular and inter- modular relations within and between foreground things and background stuff classes, and proposes a Biddirectional Graph Connection Module to diffuse information across branches in a learnable fashion.
SpatialFlow: Bridging All Tasks for Panoptic Segmentation
TLDR
A location-aware and unified framework for panoptic segmentation, denoted as SpatialFlow, and achieves state-of-the-art results, which are 47.9 PQ and 62.5 PQ respectively on MS-COCO and CityscapesPanoptic benchmarks.
Panoptic Segmentation with a Joint Semantic and Instance Segmentation Network
TLDR
This method combines the predictions from a jointly trained semantic and instance segmentation network using heuristics and is faster and more memory efficient than training and predicting with two networks, as done in previous work.
SOGNet: Scene Overlap Graph Network for Panoptic Segmentation
TLDR
This study aims to model overlap relations among instances and resolve them for panoptic segmentation, Inspired by scene graph representation, and forms the overlapping problem as a simplified case, named scene overlap graph.
DeeperLab: Single-Shot Image Parser
TLDR
The proposed DeeperLab image parser performs whole image parsing with a significantly simpler, fully convolutional approach that jointly addresses the semantic and instance segmentation tasks in a single-shot manner, resulting in a streamlined system that better lends itself to fast processing.
...
...