• Corpus ID: 235742717

Predicate correlation learning for scene graph generation

  title={Predicate correlation learning for scene graph generation},
  author={Lei Tao and Li Mi and Nannan Li and Xianhang Cheng and Yaosi Hu and Zhenzhong Chen},
For a typical Scene Graph Generation (SGG) method, there is often a large gap in the performance of the predicates’ head classes and tail classes. This phenomenon is mainly caused by the semantic overlap between different predicates as well as the long-tailed data distribution. In this paper, a Predicate Correlation Learning (PCL) method for SGG is proposed to address the above two problems by taking the correlation between predicates into consideration. To describe the semantic overlap between… 

Figures and Tables from this paper


PCPL: Predicate-Correlation Perception Learning for Unbiased Scene Graph Generation
A novel Predicate-Correlation Perception Learning scheme to adaptively seek out appropriate loss weights by directly perceiving and utilizing the correlation among predicate classes is proposed, which significantly outperforms previous state-of-the-art methods.
Relation Regularized Scene Graph Generation.
A relation regularized network (R2-Net) is proposed, which can predict whether there is a relationship between two objects and encode this relation into object feature refinement and better SGG.
Knowledge-Embedded Routing Network for Scene Graph Generation
This work finds that the statistical correlations between object pairs and their relationships can effectively regularize semantic space and make prediction less ambiguous, and thus well address the unbalanced distribution issue.
Scene Graph Generation With Hierarchical Context
The analysis shows that spatial correlations between objects, focused regions of objects, and global hints related to the relations have strong influences in relation prediction and contradiction elimination.
Scene Graph Generation from Objects, Phrases and Region Captions
This work proposes a novel neural network model, termed as Multi-level Scene Description Network (denoted as MSDN), to solve the three vision tasks jointly in an end-to-end manner and shows the joint learning across three tasks with the proposed method can bring mutual improvements over previous models.
Factorizable Net: An Efficient Subgraph-based Framework for Scene Graph Generation
A subgraph-based connection graph is proposed to concisely represent the scene graph during the inference to improve the efficiency of scene graph generation and outperforms the state-of-the-art method in both accuracy and speed.
Unbiased Scene Graph Generation From Biased Training
A novel SGG framework based on causal inference but not the conventional likelihood is presented, which uses Total Direct Effect (TDE) as the proposed final predicate score for unbiased SGG and can be widely applied in the community who seeks unbiased predictions.
Neural Motifs: Scene Graph Parsing with Global Context
This work analyzes the role of motifs: regularly appearing substructures in scene graphs and introduces Stacked Motif Networks, a new architecture designed to capture higher order motifs in scene graph graphs that improves on the previous state-of-the-art by an average of 3.6% relative improvement across evaluation settings.
GPS-Net: Graph Property Sensing Network for Scene Graph Generation
A novel message passing module that augments the node feature with node-specific contextual information and encodes the edge direction information via a tri-linear model is proposed that achieves state-of-the-art performance on three popular databases: VG, OI, and VRD by significant gains under various settings and metrics.
Visual Relationship Detection with Internal and External Linguistic Knowledge Distillation
This work uses knowledge of linguistic statistics to regularize visual model learning and suggests that with this linguistic knowledge distillation, the model outperforms the state-of- the-art methods significantly, especially when predicting unseen relationships.