• Corpus ID: 237213678

Exploiting Scene Graphs for Human-Object Interaction Detection

  title={Exploiting Scene Graphs for Human-Object Interaction Detection},
  author={Tao He and Lianli Gao and Jingkuan Song and Yuan-Fang Li},
Human-Object Interaction (HOI) detection is a fundamental visual task aiming at localizing and recognizing interactions between humans and objects. Existing works focus on the visual and linguistic features of the humans and objects. However, they do not capitalise on the high-level and semantic relationships present in the image, which provides crucial contextual and detailed relational knowledge for HOI inference. We propose a novel method to exploit this information, through the scene graph… 

Figures and Tables from this paper


Contextual Heterogeneous Graph Network for Human-Object Interaction Detection
A heterogeneous graph network that models humans and objects as different kinds of nodes and incorporates intra- class messages between homogeneous nodes and inter-class messages between heterogeneous nodes is proposed and exploited to improve the learning.
Learning to Detect Human-Object Interactions
Experiments demonstrate that the proposed Human-Object Region-based Convolutional Neural Networks (HO-RCNN), by exploiting human-object spatial relations through Interaction Patterns, significantly improves the performance of HOI detection over baseline approaches.
iCAN: Instance-Centric Attention Network for Human-Object Interaction Detection
This paper proposes an instance-centric attention module that learns to dynamically highlight regions in an image conditioned on the appearance of each instance and allows an attention-based network to selectively aggregate features relevant for recognizing HOIs.
Learning Human-Object Interaction Detection Using Interaction Points
This paper proposes a novel fully-convolutional approach that directly detects the interactions between human-object pairs and predicts interaction points, which directly localize and classify the inter-action.
Detecting and Recognizing Human-Object Interactions
A novel model is proposed that learns to predict an action-specific density over target object locations based on the appearance of a detected person and efficiently infers interaction triplets in a clean, jointly trained end-to-end system the authors call InteractNet.
Transferable Interactiveness Knowledge for Human-Object Interaction Detection
The core idea is to exploit an Interactiveness Network to learn the general interactiveness knowledge from multiple HOI datasets and perform Non-Interaction Suppression before HOI classification in inference and extensively evaluate the proposed method on HICO-DET and V-COCO datasets.
Amplifying Key Cues for Human-Object-Interaction Detection
Two methods to amplify key cues in the image and a method to combine these and other cues when considering the interaction between a human and an object, which exceeds prior HOI methods across standard benchmarks by a considerable margin.
VSGNet: Spatial Attention Network for Detecting Human Object Interactions Using Graph Convolutions
The proposed Visual-Spatial-Graph Network (VSGNet) architecture extracts visual features from the human-object pairs, refines the features with spatial configurations of the pair, and utilizes the structural connections between the pair via graph convolutions.
Learning Human-Object Interactions by Graph Parsing Neural Networks
This paper addresses the task of detecting and recognizing human-object interactions (HOI) in images and videos. We introduce the Graph Parsing Neural Network (GPNN), a framework that incorporates
Polysemy Deciphering Network for Human-Object Interaction Detection
A novel Polysemy Deciphering Network (PD-Net), which decodes the visual polysemy of verbs for HOI detection in three ways, and achieves the best performance on both HICO-DET and V-COCO datasets.