Reciprocal Learning Networks for Human Trajectory Prediction

@article{Sun2020ReciprocalLN,
  title={Reciprocal Learning Networks for Human Trajectory Prediction},
  author={Hao Sun and Zhiqun Zhao and Zhihai He},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2020},
  pages={7414-7423}
}
  • Hao SunZhiqun ZhaoZhihai He
  • Published 9 April 2020
  • Computer Science
  • 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
We observe that the human trajectory is not only forward predictable, but also backward predictable. Both forward and backward trajectories follow the same social norms and obey the same physical constraints with the only difference in their time directions. Based on this unique property, we develop a new approach, called reciprocal learning, for human trajectory prediction. Two networks, forward and backward prediction networks, are tightly coupled, satisfying the reciprocal constraint, which… 
[PDF] Semantic Reader
29 Citations
Highly Influential Citations
2
Background Citations
21
Methods Citations
13
Results Citations
1

Figures and Tables from this paper

29 Citations

Safety-compliant Generative Adversarial Networks for Human Trajectory Forecasting

SGANv2 is introduced: an improved safety-compliant SGAN architecture equipped with spatio-temporal interaction modelling and a transformer-based discriminator design that improves temporal sequence modelling and helps to learn the human social interactions better.

On Adversarial Robustness of Trajectory Prediction for Autonomous Vehicles

This work studies the adversarial robustness of trajectory prediction models by proposing a new adversarial attack that perturbs normal vehicle trajectories to maximize the prediction error and explores possible mitigation techniques via data augmentation and trajectory smoothing.

Stochastic Trajectory Prediction via Motion Indeterminacy Diffusion

This paper presents a new framework to formulate the trajectory prediction task as a reverse process of motion indeterminacy diffusion (MID), in which the history behavior information and the social interactions are encoded as a state embedding and a Transformer-based diffusion model is devised to capture the temporal dependencies of trajectories.

Disentangled Multi-Relational Graph Convolutional Network for Pedestrian Trajectory Prediction

A novel disentangled multi-scale aggregation to better represent social interactions, among pedestrians on a weighted graph, and a global temporal aggregation to alleviate accumulated errors for pedestrians changing their directions are introduced.

Decoder Fusion RNN: Context and Interaction Aware Decoders for Trajectory Prediction

This paper proposes Decoder Fusion RNN (DF-RNN), a recurrent, attention-based approach for motion forecasting that is composed of a recurrent behavior encoder, an inter-agent multi-headed attention module, and a context-aware decoder.

Self-Constrained Inference Optimization on Structural Groups for Human Pose Estimation

This work develops a self-constrained prediction-verification network to characterize and learn the structural correlation between keypoints during training, and demonstrates that the proposed method can significantly improve the pose estimation results.

SFMGNet: A Physics-based Neural Network To Predict Pedestrian Trajectories

Initial results suggest that, even when solely trained on a synthetic dataset, the model can predict realistic and interpretable trajectories with better than state-of-the-art accuracy.

LatentFormer: Multi-Agent Transformer-Based Interaction Modeling and Trajectory Prediction

LatentFormer, a transformerbased model for predicting future vehicle trajectories, is proposed that leverages a novel technique for modeling interactions among dynamic objects in the scene and achieves state-of-the-art performance and improves upon trajectory metrics by up to 40%.

PedFormer: Pedestrian Behavior Prediction via Cross-Modal Attention Modulation and Gated Multitask Learning

This work proposes a novel framework that relies on different data modalities to predict future trajectories and crossing actions of pedestrians from an ego-centric perspective and demonstrates that this model improves state-of-the-art in trajectory and action prediction by up to 22% and 13% respectively on various metrics.

Pedestrian Behavior Prediction via Multitask Learning and Categorical Interaction Modeling

A multitask learning framework that simultaneously predicts trajectories and actions of pedestrians by relying on multimodal data is proposed and achieves state-of-the-art performance and improves trajectory and action prediction by up to 22% and 6% respectively.

References

SHOWING 1-10 OF 42 REFERENCES

Social LSTM: Human Trajectory Prediction in Crowded Spaces

This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.

Social GAN: Socially Acceptable Trajectories with Generative Adversarial Networks

A recurrent sequence-to-sequence model observes motion histories and predicts future behavior, using a novel pooling mechanism to aggregate information across people, and outperforms prior work in terms of accuracy, variety, collision avoidance, and computational complexity.

Peeking Into the Future: Predicting Future Person Activities and Locations in Videos

An end-to-end, multi-task learning system utilizing rich visual features about human behavioral information and interaction with their surroundings is proposed, providing the first empirical evidence that joint modeling of paths and activities benefits future path prediction.

Context-Aware Trajectory Prediction

This work proposes a “context-aware” recurrent neural network LSTM model, which can learn and predict human motion in crowded spaces such as a sidewalk, a museum or a shopping mall, and evaluates the model on a public pedestrian datasets.

DESIRE: Distant Future Prediction in Dynamic Scenes with Interacting Agents

The proposed Deep Stochastic IOC RNN Encoder-decoder framework, DESIRE, for the task of future predictions of multiple interacting agents in dynamic scenes significantly improves the prediction accuracy compared to other baseline methods.

CAR-Net: Clairvoyant Attentive Recurrent Network

A Clairvoyant Attentive Recurrent Network (CAR-Net) that learns where to look in a large image of the scene when solving the path prediction task, and shows CAR-Net’s ability to generalize to unseen scenes.

Explaining and Harnessing Adversarial Examples

It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets.

Intriguing properties of neural networks

It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks.

Group LSTM: Group Trajectory Prediction in Crowded Scenarios

This work proposes a novel approach to predict future trajectories in crowded scenes, at the group level, by exploiting the motion coherency and cluster trajectories that have similar motion trends, so pedestrians within the same group can be well segmented.

Scene-LSTM: A Model for Human Trajectory Prediction

A human movement trajectory prediction system that incorporates the scene information as well as human movement trajectories (Pedestrian movement LSTM) in the prediction process within static crowded scenes and reduces the location displacement errors compared to related methods and specifically about 80% reduction compared to social interaction methods.