PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

@article{Rhinehart2019PRECOGPC,
  title={PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings},
  author={Nicholas Rhinehart and Rowan Thomas McAllister and Kris Kitani and Sergey Levine},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={2821-2830}
}
For autonomous vehicles (AVs) to behave appropriately on roads populated by human-driven vehicles, they must be able to reason about the uncertain intentions and decisions of other drivers from rich perceptual information. [] Key Method We train our model on real and simulated data to forecast vehicle trajectories given past positions and LIDAR. Our evaluation shows that our model is substantially more accurate in multi-agent driving scenarios compared to existing state-of-the-art. Beyond its general ability…
[PDF] Semantic Reader
239 Citations
Highly Influential Citations
33
Background Citations
155
Methods Citations
88
Results Citations
7

239 Citations

Scene Transformer: A unified architecture for predicting multiple agent trajectories

A model for predicting the behavior of all agents jointly, producing consistent futures that account for interactions between agents is formulated, and it is shown that the model can unify a variety of motion prediction tasks from joint motion predictions to conditioned prediction.

Multiple Trajectory Prediction of Moving Agents with Memory Augmented Networks.

This paper proposes MANTRA, a model that exploits memory augmented networks to effectively predict multiple trajectories of other agents, observed from an egocentric perspective, and shows how once trained the system can continuously improve by ingesting novel patterns.

Heterogeneous-Agent Trajectory Forecasting Incorporating Class Uncertainty

HAICU, a method for heterogeneous-agent trajectory forecasting that explicitly incorporates agents' class probabilities, is presented, demonstrating that incorporating class probabilities in trajectory forecasting significantly improves performance in the face of uncertainty, and enables new forecasting capabilities such as counterfactual predictions.

Multiple Futures Prediction

A probabilistic framework that efficiently learns latent variables to jointly model the multi-step future motions of agents in a scene and can be used for planning via computing a conditional probability density over the trajectories of other agents given a hypothetical rollout of the ego agent.

MIDAS: Multi-agent Interaction-aware Decision-making with Adaptive Strategies for Urban Autonomous Navigation

MIDAS uses an attention mechanism to handle an arbitrary number of other agents and includes a "driver-type" parameter to learn a single policy that works across different planning objectives and is safer and more efficient than existing approaches to interaction-aware decision-making.

Trajectron++: Multi-Agent Generative Trajectory Forecasting With Heterogeneous Data for Control

Trajectron++ is a modular, graph-structured recurrent model that forecasts the trajectories of a general number of agents with distinct semantic classes while incorporating heterogeneous data (e.g. semantic maps and camera images) and is designed to be tightly integrated with robotic planning and control frameworks.

Instance-Aware Predictive Navigation in Multi-Agent Environments

The Instance-Aware Predictive Control approach is proposed, which forecasts interactions between agents as well as future scene structures, and a sequential action sampling strategy is designed to better leverage predicted states on both scene-level and instance-level.

Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data

Trajectron++ is a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents while incorporating agent dynamics and heterogeneous data and outperforming a wide array of state-of-the-art deterministic and generative methods.

PredictionNet: Real-Time Joint Probabilistic Traffic Prediction for Planning, Control, and Simulation

Predicting the future motion of traffic agents is crucial for safe and efficient autonomous driving. To this end, we present PredictionNet, a deep neural network (DNN) that predicts the motion of all…

PLOP: Probabilistic Polynomial Objects trajectory Prediction for autonomous driving

The model processes ego vehicle front-facing camera images and bird-eye view grid, computed from Lidar point clouds, with detections of past and present objects, in order to generate multiple trajectories for both ego vehicle and its neighbors through a probabilistic framework.
...

References

SHOWING 1-10 OF 38 REFERENCES

Multi-Modal Trajectory Prediction of Surrounding Vehicles with Maneuver based LSTMs

This paper presents an LSTM model for interaction aware motion prediction of surrounding vehicles on freeways and assigns confidence values to maneuvers being performed by vehicles and outputs a multi-modal distribution over future motion based on them.

Multimodal Probabilistic Model-Based Planning for Human-Robot Interaction

The approach is to learn multimodal probability distributions over future human actions from a dataset of human-human exemplars and perform real-time robot policy construction in the resulting environment model through massively parallel sampling of human responses to candidate robot action sequences.

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

The proposed Deep Stochastic IOC RNN Encoder-decoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes significantly improves the prediction accuracy compared to other baseline methods.

Multi-Agent Tensor Fusion for Contextual Trajectory Prediction

  • Tianyang ZhaoYifei Xu Y. Wu
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
This work encodes multiple agents' past trajectories and the scene context into a Multi-Agent Tensor, then applies convolutional fusion to capture multiagent interactions while retaining the spatial structure of agents and thescene context.

Deep Imitative Models for Flexible Inference, Planning, and Control

This paper proposes Imitative Models, probabilistic predictive models of desirable behavior able to plan interpretable expert-like trajectories to achieve specified goals, and derives families of flexible goal objectives that can be used to successfully direct behavior.

Generative Modeling of Multimodal Multi-Human Behavior

This work presents a methodology for modeling and predicting human behavior in settings with $N$ humans interacting in highly multimodal scenarios (i.e. where there are many possible highly-distinct futures), and demonstrates the performance of such a modeling approach in modeling basketball player trajectories, a highly multi-human scenario which serves as a proxy for many robotic applications.

Social LSTM: Human Trajectory Prediction in Crowded Spaces

This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.

Context-Aware Trajectory Prediction

This work proposes a “context-aware” recurrent neural network LSTM model, which can learn and predict human motion in crowded spaces such as a sidewalk, a museum or a shopping mall, and evaluates the model on a public pedestrian datasets.

Hierarchical Game-Theoretic Planning for Autonomous Vehicles

This paper introduces a novel game-theoretic trajectory planning algorithm for autonomous driving that enables real-time performance by hierarchically decomposing the underlying dynamic game into a long-horizon “strategic” game with simplified dynamics and full information structure, and a short-Horizon ‘tactical’ game with full dynamics and a simplified information structure.

Stochastic Prediction of Multi-Agent Interactions from Partial Observations

We present a method that learns to integrate temporal information, from a learned dynamics model, with ambiguous visual information, from a learned vision model, in the context of interacting agents.…