The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation

@inproceedings{Wang2020TheDI,
  title={The Devil is in Classification: A Simple Framework for Long-tail Instance Segmentation},
  author={Tao Wang and Yu Li and Bingyi Kang and Junnan Li and Jun Hao Liew and Sheng Tang and Steven C. H. Hoi and Jiashi Feng},
  booktitle={European Conference on Computer Vision},
  year={2020}
}
  • Tao WangYu Li Jiashi Feng
  • Published in
    European Conference on…
     23 July 2020
  • Computer Science, Environmental Science
Most existing object instance detection and segmentation models only work well on fairly balanced benchmarks where per-category training sample numbers are comparable, such as COCO. They tend to suffer performance drop on realistic datasets that are usually long-tailed. This work aims to study and address such open challenges. Specifically, we systematically investigate performance drop of the state-of-the-art two-stage instance segmentation model Mask R-CNN on the recent long-tail LVIS dataset… 
[PDF] Semantic Reader
70 Citations
Highly Influential Citations
12
Background Citations
37
Methods Citations
27
Results Citations
1

70 Citations

Classification Calibration for Long-tail Instance Segmentation

This report investigates the performance drop phenomenon of state-of-the-art two-stage instance segmentation models when processing extreme long-tail training data based on the LVIS dataset, and finds a major cause is the inaccurate classification of object proposals.

Long-Tail Instance Segmentation Based on Memory Bank and Confidence Calibration

Anobject-centric memory bank is used to establish an object-centric storage strategy that can solve the imbalance problem with respect to categories and improves segmentation accuracy.

A Benchmark of Long-tailed Instance Segmentation with Noisy Labels (Short Version)

This paper considers the instance segmentation task on a long-tailed dataset, which contains label noise, i.e., some of the annotations are incorrect, and proposes a new datasets, which is a large vocabulary long-tails dataset containing label noise for instance segmentations.

Long-tailed Instance Segmentation using Gumbel Optimized Loss

This paper develops a Gumbel Optimized Loss ( GOL), for long-tailed detection and segmentation, which significantly outperforms the best state-of-the-art method and boosts the overall segmentation and detection, particularly improving detection of rare classes on LVIS dataset.

Seesaw Loss for Long-Tailed Instance Segmentation

This work proposes Seesaw Loss to dynamically re-balance gradients of positive and negative samples for each category, with two complementary factors, i.e., mitigation factor and compensation factor, which obtains significant gains over Cross-Entropy Loss and achieves state-of-the-art performance on LVIS dataset without bells and whistles.

Long-tail Detection with Effective Class-Margins

A theoretical understanding of the long-trail detection problem is provided and it is shown how the commonly used mean average precision evaluation metric on an unknown test set is bound by a margin-based binary classification error on a long-tailed object detection training set.

Relieving Long-tailed Instance Segmentation via Pairwise Class Balance

This paper proposes a novel Pairwise Class Balance (PCB) method, built upon a confusion matrix which is updated during training to accumulate the ongoing prediction preferences, and generates fightback soft labels for regularization during training.

Learning with Free Object Segments for Long-Tailed Instance Segmentation

This paper investigates the simi-larity among object-centric images of the same class to propose candidate segments of foreground instances, followed by a novel ranking of segment quality and proposes a simple and scalable framework F REE S EG for ex-tracting and leveraging these “free” object segments to facilitate model training.

Trapped in texture bias? A large scale comparison of deep instance segmentation

YOLACT++, SOTR and SOLOv2 are significantly more robust to out-of-distribution texture than other frameworks and it is shown that deeper and dynamic architectures improve robustness whereas training schedules, data augmentation and pre-training have only a minor impact.

FASA: Feature Augmentation and Sampling Adaptation for Long-Tailed Instance Segmentation

This work proposes a simple yet effective method, Feature Augmentation and Sampling Adaptation (FASA), that addresses the data scarcity issue by augmenting the feature space especially for rare classes by eliminating the need for inter-class transfer learning.
...

References

SHOWING 1-10 OF 40 REFERENCES

Classification Calibration for Long-tail Instance Segmentation

This report investigates the performance drop phenomenon of state-of-the-art two-stage instance segmentation models when processing extreme long-tail training data based on the LVIS dataset, and finds a major cause is the inaccurate classification of object proposals.

Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation

This paper proposes a simple and scalable detection algorithm that improves mean average precision (mAP) by more than 30% relative to the previous best result on VOC 2012 -- achieving a mAP of 53.3%.

RepMet: Representative-Based Metric Learning for Classification and Few-Shot Object Detection

This work proposes a new method for DML that simultaneously learns the backbone network parameters, the embedding space, and the multi-modal distribution of each of the training categories in that space, in a single end-to-end training process.

Mask Scoring R-CNN

This paper proposes Mask Scoring R-CNN which contains a network block to learn the quality of the predicted instance masks and calibrates the misalignment between mask quality and mask score, and improves instance segmentation performance by prioritizing more accurate mask predictions during COCO AP evaluation.

Hybrid Task Cascade for Instance Segmentation

This work proposes a new framework, Hybrid Task Cascade (HTC), which differs in two important aspects: (1) instead of performing cascaded refinement on these two tasks separately, it interweaves them for a joint multi-stage processing; (2) it adopts a fully convolutional branch to provide spatial context, which can help distinguishing hard foreground from cluttered background.

Decoupling Representation and Classifier for Long-Tailed Recognition

It is shown that it is possible to outperform carefully designed losses, sampling strategies, even complex modules with memory, by using a straightforward approach that decouples representation and classification.

Learning Deep Representation for Imbalanced Classification

The representation learned by this approach, when combined with a simple k-nearest neighbor (kNN) algorithm, shows significant improvements over existing methods on both high- and low-level vision classification tasks that exhibit imbalanced class distribution.

Few-Shot Object Detection via Feature Reweighting

This work develops a few-shot object detector that can learn to detect novel objects from only a few annotated examples, using a meta feature learner and a reweighting module within a one-stage detection architecture.

Path Aggregation Network for Instance Segmentation

Path Aggregation Network (PANet) is proposed aiming at boosting information flow in proposal-based instance segmentation framework by enhancing the entire feature hierarchy with accurate localization signals in lower layers by bottom-up path augmentation.

Learning Imbalanced Datasets with Label-Distribution-Aware Margin Loss

A theoretically-principled label-distribution-aware margin (LDAM) loss motivated by minimizing a margin-based generalization bound is proposed that replaces the standard cross-entropy objective during training and can be applied with prior strategies for training with class-imbalance such as re-weighting or re-sampling.