Importance is in your attention: agent importance prediction for autonomous driving

  title={Importance is in your attention: agent importance prediction for autonomous driving},
  author={Christopher Hazard and Akshay Bhagat and Balarama Raju Buddharaju and Zhongtao Liu and Yunming Shao and Lu Lu and Sammy Omari and Henggang Cui},
  journal={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
Trajectory prediction is an important task in autonomous driving. State-of-the-art trajectory prediction models often use attention mechanisms to model the interaction between agents. In this paper, we show that the attention information from such models can also be used to measure the importance of each agent with respect to the ego vehicle’s future planned trajectory. Our experiment results on the nuPlans dataset show that our method can effectively find and rank surrounding agents by their… 

Figures and Tables from this paper



Agent Prioritization for Autonomous Navigation

This work proposes a system to rank agents around an autonomous vehicle (AV) in real time, and shows the utility of combining learned features, via a convolutional neural network, with engineered features designed to capture domain knowledge.

Scene Transformer: A unified multi-task model for behavior prediction and planning

This work demonstrates that formulating the problem of behavior prediction in a unified architecture with a masking strategy may allow us to have a single model that can perform multiple motion prediction and planning related tasks effectively.

THOMAS: Trajectory Heatmap Output with learned Multi-Agent Sampling

It is highlighted that generating scene-consistent predictions goes beyond the mere generation of collision-free trajectories, so a learnable trajectory recombination model is proposed that takes as input a set of predicted trajectories for each agent and outputs its consistent reordered recombination.

VectorNet: Encoding HD Maps and Agent Dynamics From Vectorized Representation

  • Jiyang GaoChen Sun C. Schmid
  • Computer Science
    2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2020
VectorNet is introduced, a hierarchical graph neural network that first exploits the spatial locality of individual road components represented by vectors and then models the high-order interactions among all components and obtains state-of-the-art performance on the Argoverse dataset.

Multimodal Motion Prediction with Stacked Transformers

A novel network architecture based on stacked transformers is designed to model the multimodality at feature level with a set of fixed independent proposals, and achieves the state-of-the-art performance on motion prediction.

nuPlan: A closed-loop ML-based planning benchmark for autonomous vehicles

This work proposes the world’s first closed-loop ML-based planning benchmark for autonomous driving by introducing a largescale driving dataset, lightweight closed- loop simulator, and motion-planning-specific metrics.

GOHOME: Graph-Oriented Heatmap Output for future Motion Estimation

GOHOME, a method leveraging graph representations of the High Definition Map and sparse projections to generate a heatmap output representing the future position probability distribution for a given agent in a traffic scene, yields an unconstrained 2D grid representation of agent future possible locations, allowing inherent multimodality and a measure of the uncertainty of the prediction.

Learning Lane Graph Representations for Motion Forecasting

A motion forecasting model that exploits a novel structured map representation as well as actor-map interactions to predict accurate and realistic multi-modal trajectories and significantly outperforms the state-of-the-art on the large scale Argoverse motion forecasting benchmark.

Attention is All you Need

A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Graph Attention Networks

We present graph attention networks (GATs), novel neural network architectures that operate on graph-structured data, leveraging masked self-attentional layers to address the shortcomings of prior