RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting

  title={RAIN: Reinforced Hybrid Attention Inference Network for Motion Forecasting},
  author={Jiachen Li and F. Yang and Hengbo Ma and Srikanth Malla and Masayoshi Tomizuka and Chiho Choi},
  journal={2021 IEEE/CVF International Conference on Computer Vision (ICCV)},
  • Jiachen LiF. Yang Chiho Choi
  • Published 3 August 2021
  • Computer Science
  • 2021 IEEE/CVF International Conference on Computer Vision (ICCV)
Motion forecasting plays a significant role in various domains (e.g., autonomous driving, human-robot interaction), which aims to predict future motion sequences given a set of historical observations. However, the observed elements may be of different levels of importance. Some information may be irrelevant or even distracting to the forecasting in certain situations. To address this issue, we propose a generic motion forecasting framework (named RAIN) with dynamic key information selection… 

Multi-Person Extreme Motion Prediction

A novel cross interaction attention mechanism that exploits historical information of both persons, and learns to predict cross dependencies between the two pose sequences to predict the future motion of two interacted persons given two sequences of their past skeletons.

Multi-Objective Diverse Human Motion Prediction with Knowledge Distillation

This work designs a prediction frame-work that can balance the accuracy sampling and diversity sampling during the testing phase and proposes a multi-objective conditional variational inference prediction model, which includes a short-term oracle to encourage the prediction framework to explore more diverse future motions.

Dyadic Human Motion Prediction

This paper introduces a motion prediction framework that explicitly reasons about the interactions of two observed subjects and introduces a pairwise attention mechanism that models the mutual dependencies in the motion history of the two subjects.

StarNet: Joint Action-Space Prediction with Star Graphs and Implicit Global-Frame Self-Attention

This work presents a novel multi-modal multi-agent trajectory prediction architecture, focusing on map and interaction modeling using graph representation, and extends this architecture to joint scene-level prediction, which produces multiple agents’ predictions simultaneously.

LOKI: Long Term and Key Intentions for Trajectory Prediction

The LOKI dataset is created to discover several factors that may affect intention, including i) agent’s own will, ii) social interactions, iii) environmental constraints, and iv) contextual information, and a model that jointly performs trajectory and intention prediction is proposed, showing that recurrently reasoning about intention can assist with trajectory prediction.

HiT-DVAE: Human Motion Generation via Hierarchical Transformer Dynamical VAE

This paper proposes Hierarchical Transformer Dynamical Variational Variational Autoencoder, HiT-DVAE, which implements auto-regressive generation with transformer-like attention mechanisms, thus enabling the generative model to learn a more complex and time-varying latent space as well as diverse and realistic human motions.

Important Object Identification with Semi-Supervised Learning for Autonomous Driving

This work proposes a novel approach for important object identification in egocentric driving scenarios with relational reasoning on the objects in the scene with a semi-supervised learning pipeline to enable the model to learn from unlimited unlabeled data.

Hierarchical Latent Structure for Multi-Modal Vehicle Trajectory Forecasting

Experimental results show that the proposed VAE-based trajectory forecasting model is not only capable of generating clear multi-modal trajectory distributions but also outperforms the state-of-the-art (SOTA) models in terms of prediction accuracy.

Adaptive Environment Modeling Based Reinforcement Learning for Collision Avoidance in Complex Scenes

A novel adaptive environment model based collision avoidance reinforcement learning (i.e., AEMCARL) framework for an unmanned robot to achieve collision-free motions in challenging navigation scenarios.

EvolveHypergraph: Group-Aware Dynamic Relational Reasoning for Trajectory Prediction

This paper proposes a group-aware relational reasoning approach with explicit inference of the underlying dynamically evolving relational structures, and demonstrates its effectiveness for multi-agent trajectory prediction.



DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

The proposed Deep Stochastic IOC RNN Encoder-decoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes significantly improves the prediction accuracy compared to other baseline methods.

Multiple Futures Prediction

A probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene and can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the ego agent.

Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control

Trajectron++ is a modular, graph-structured recurrent model that forecasts the trajectories of a general number of agents with distinct semantic classes while incorporating heterogeneous data (e.g. semantic maps and camera images) and is designed to be tightly integrated with robotic planning and control frameworks.

Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking

A generic generative neural system (called STG-DAT) for multi-agent trajectory prediction involving heterogeneous agents takes a step forward to explicit interaction modeling by incorporating relational inductive biases with a dynamic graph representation and leverages both trajectory and scene context information.

EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning

This paper proposes a generic trajectory forecasting framework with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents and introduces a double-stage training pipeline which not only improves training efficiency and accelerates convergence, but also enhances model performance.

History Repeats Itself: Human Motion Prediction via Motion Attention

An attention-based feed-forward network is introduced that explicitly leverages the observation that human motion tends to repeat itself to capture motion attention to capture the similarity between the current motion context and the historical motion sub-sequences.

Continual Multi-Agent Interaction Behavior Prediction With Conditional Generative Memory

This work proposes a multi-agent interaction behavior prediction framework with a graph-neural-network-based conditional generative memory system to mitigate catastrophic forgetting and empirically shows that several approaches in literature indeed suffer from catastrophic forgetting.

Multi-Agent Driving Behavior Prediction across Different Scenarios with Self-Supervised Domain Knowledge

A graph-neural-network-based framework for multi-agent interaction-aware trajectory prediction is introduced and human's prior knowledge such as the comprehension of pairwise relations between agents and pairwise context information extracted by self-supervised learning approaches are proposed to attain an effective Frenét-based representation.

Conditional Generative Neural System for Probabilistic Trajectory Prediction

This paper proposes a conditional generative neural system (CGNS) for probabilistic trajectory prediction to approximate the data distribution, with which realistic, feasible and diverse future trajectory hypotheses can be sampled.

Social LSTM: Human Trajectory Prediction in Crowded Spaces

This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.