Interaction Modeling with Multiplex Attention

  title={Interaction Modeling with Multiplex Attention},
  author={Fan-Yun Sun and Isaac Kauvar and Ruohan Zhang and Jiachen Li and Mykel J. Kochenderfer and Jiajun Wu and Nick Haber},
Modeling multi-agent systems requires understanding how agents interact. Such systems are often difficult to model because they can involve a variety of types of interactions that layer together to drive rich social behavioral dynamics. Here we introduce a method for accurately modeling multi-agent systems. We present Interaction Modeling with Multiplex Attention (IMMA), a forward prediction model that uses a multiplex latent graph to represent multiple independent types of interactions and… 
1 Citations

Learning Heterogeneous Interaction Strengths by Trajectory Prediction with Graph Neural Network

The RAIN model with the PA mechanism accurately infers continuous interaction strengths for simulated physical systems in an unsupervised manner, and successfully predicts trajectories from motion capture data with an interpretable interaction graph, demonstrating the virtue of modeling unknown dynamics with continuous weights.



Relational Forward Models for Multi-Agent Learning

Relational Forward Models (RFM) for multi-agent learning are introduced, networks that can learn to make accurate predictions of agents' future behavior in multi- agent environments.

EvolveGraph: Multi-Agent Trajectory Prediction with Dynamic Relational Reasoning

This paper proposes a generic trajectory forecasting framework with explicit relational structure recognition and prediction via latent interaction graphs among multiple heterogeneous, interactive agents and introduces a double-stage training pipeline which not only improves training efficiency and accelerates convergence, but also enhances model performance.

Generating Multi-Agent Trajectories using Programmatic Weak Supervision

This work presents a hierarchical framework that can effectively learn sequential generative models for capturing coordinated multi-agent trajectory behavior, such as offensive basketball gameplay, and is inspired by recent work on leveraging programmatically produced weak labels, which it extends to the spatiotemporal regime.

Factorised Neural Relational Inference for Multi-Interaction Systems

This work considers an unsupervised explicit model, the NRI model, and makes a series of representational adaptations and physically motivated changes that factorise the inferred latent interaction graph into a multiplex graph, allowing each layer to encode for a different interaction-type.

Coordinated Multi-Agent Imitation Learning

It is shown that having a coordination model to infer the roles of players yields substantially improved imitation loss compared to conventional baselines, and the method integrates unsupervised structure learning with conventional imitation learning.

Relational Graph Learning for Crowd Navigation

This work presents a relational graph learning approach for robotic crowd navigation using model-based deep reinforcement learning that plans actions by looking into the future and uses a Graph Convolutional Network to encode higher-order interactions in each agent’s state representation.

Neural Relational Inference for Interacting Systems

The NRI model is introduced: an unsupervised model that learns to infer interactions while simultaneously learning the dynamics purely from observational data, in the form of a variational auto-encoder.

Spatio-Temporal Graph Dual-Attention Network for Multi-Agent Prediction and Tracking

A generic generative neural system (called STG-DAT) for multi-agent trajectory prediction involving heterogeneous agents takes a step forward to explicit interaction modeling by incorporating relational inductive biases with a dynamic graph representation and leverages both trajectory and scene context information.

Crowd-Robot Interaction: Crowd-Aware Robot Navigation With Attention-Based Deep Reinforcement Learning

This work proposes to rethink pairwise interactions with a self-attention mechanism, and jointly model Human-Robot as well as Human-Human interactions in the deep reinforcement learning framework, and captures the Human- human interactions occurring in dense crowds that indirectly affects the robot’s anticipation capability.

Learning World Graphs to Accelerate Hierarchical Reinforcement Learning

A thorough ablation study is performed to evaluate the proposed graph abstraction over the environment structure to accelerate the learning of these tasks with significant advantages from the proposed framework over baselines that lack world graph knowledge in terms of performance and efficiency.