MTP: Multi-hypothesis Tracking and Prediction for Reduced Error Propagation

  title={MTP: Multi-hypothesis Tracking and Prediction for Reduced Error Propagation},
  author={Xinshuo Weng and B. Ivanovic and Marco Pavone},
  journal={2022 IEEE Intelligent Vehicles Symposium (IV)},
There has been tremendous progress in the development of individual modules of the standard perception-prediction-planning robot autonomy stack. However, the principled integration of these modules has received less attention, particularly in terms of cascading errors. In this work, we both characterize and address the problem of cascading errors, focusing on the coupling between tracking and prediction. First, we comprehensively evaluate the impact of tracking errors on prediction performance… 

Figures and Tables from this paper

Whose Track Is It Anyway? Improving Robustness to Tracking Errors with Affinity-based Trajectory Prediction

This work proposes a new prediction paradigm that uses detections and their affinity matrices across frames as inputs, removing the need for error- prone data association during tracking, and reduces overall prediction errors by up to 57.9%.

Task-Relevant Failure Detection for Trajectory Predictors in Autonomous Vehicles

A probabilistic run-time monitor that detects when a “harmful” prediction failure occurs, i.e., a task-relevant failure detector is developed by propagating trajectory prediction errors to the planning cost to reason about their impact on the AV.

Exploring Map-based Features for Efficient Attention-based Vehicle Motion Prediction

This work explores how to achieve competitive performance on the Argoverse 1.0 Benchmark using efficient attention-based models, which take as input the past trajectories and map-based features from minimal map information to ensure efficient and reliable MP.

Uncertainty estimation for Cross-dataset performance in Trajectory prediction

This paper observes the performance of two of the latest state-of-the-art trajectory prediction methods across four different datasets and presents a novel method to estimate prediction uncertainty and shows how it could be used to achieve better performance across datasets.

SpOT: Spatiotemporal Modeling for 3D Object Tracking

This work reformulate tracking as a spatiotemporal problem by representing tracked objects as sequences of time-stamped points and bounding boxes over a long temporal history, and develops a holistic representation of traffic scenes that leverages both spatial and temporal information of the actors in the scene.

Target Identification via Multi ‐ View Multi ‐ Task Joint Sparse Representation

An automatic target identification method using surveillance video, which provides an effective solution for the surveillance video data, and is superior to other classical methods, qualitatively and quantitatively.

DiffStack: A Differentiable and Modular Control Stack for Autonomous Vehicles

DiffStack is presented, a differentiable and modular stack for prediction, planning, and control that leverage recent advancements in differentiable optimization to produce gradients, enabling optimization of upstream components, such as prediction, via backpropagation through planning and control.



PnPNet: End-to-End Perception and Prediction With Tracking in the Loop

This work proposes PnPNet, an end-to-end model that takes as input sequential sensor data, and outputs at each time step object tracks and their future trajectories, and shows significant improvements over the state-of-the-art with better occlusion recovery and more accurate future prediction.

PTP: Parallelized Tracking and Prediction With Graph Neural Networks and Diversity Sampling

This work proposes a parallelized framework for tracking and prediction sequentially which can propagate errors from tracking to prediction, and uses a diversity sampling function to improve the quality and diversity of the authors' forecasted trajectories.

Towards Robust Human Trajectory Prediction in Raw Videos

  • Rui YuZihan Zhou
  • Computer Science
    2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2021
This paper proposes a simple yet effective strategy to correct the tracking failures by enforcing prediction consistency over time and demonstrates that the proposed method can improve both tracking and prediction performance in challenging real-world scenarios.

Rethinking Trajectory Forecasting Evaluation

This work critically evaluates current trajectory forecasting metrics, proposing task-aware metrics as a better measure of performance in systems where prediction is being deployed, and additionally presents one example of such a metric, incorporating planningawareness within existing trajectory forecast metrics.

Multiple Hypothesis Tracking Revisited

It is demonstrated that a classical MHT implementation from the 90's can come surprisingly close to the performance of state-of-the-art methods on standard benchmark datasets, and it is shown that appearance models can be learned efficiently via a regularized least squares framework.

Trajectron++: Dynamically-Feasible Trajectory Forecasting with Heterogeneous Data

Trajectron++ is a modular, graph-structured recurrent model that forecasts the trajectories of a general number of diverse agents while incorporating agent dynamics and heterogeneous data and outperforming a wide array of state-of-the-art deterministic and generative methods.

EagerMOT: 3D Multi-Object Tracking via Sensor Fusion

This paper proposes EagerMOT, a simple tracking formulation that eagerly integrates all available object observations from both sensor modalities to obtain a well-informed interpretation of the scene dynamics and achieves state-of-the-art results across several MOT tasks on the KITTI and NuScenes datasets.

An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking

An efficient implementation of Reid's multiple hypothesis tracking (MHT) algorithm is presented in which the k-best hypotheses are determined in polynomial time using an algorithm due to Murly

Score refinement for confidence-based 3D multi-object tracking

This work focuses on a neglected part of the tracking system: score refinement and tracklet termination, and shows that manipulating the scores depending on time consistency while terminating the tracklets depending on the tracklet score improves tracking results.

PRECOG: PREdiction Conditioned on Goals in Visual Multi-Agent Settings

A probabilistic forecasting model of future interactions between a variable number of agents that performs both standard forecasting and the novel task of conditional forecasting, which reasons about how all agents will likely respond to the goal of a controlled agent.