SSeg-LSTM: Semantic Scene Segmentation for Trajectory Prediction

@article{Syed2019SSegLSTMSS,
  title={SSeg-LSTM: Semantic Scene Segmentation for Trajectory Prediction},
  author={Arsal Syed and Brendan Tran Morris},
  journal={2019 IEEE Intelligent Vehicles Symposium (IV)},
  year={2019},
  pages={2504-2509}
}
  • Arsal SyedB. Morris
  • Published 9 June 2019
  • Computer Science
  • 2019 IEEE Intelligent Vehicles Symposium (IV)
In this paper, we propose the use of semantic segmentation to incorporate scene information for better understanding of human motion in crowded environments. Our proposed SSeg-LSTM method leverages SegNet, which is a semantic segmentation encoder-decoder architecture, to extract semantically meaningful scene features. We then train the Social Scene LSTM (SS-LSTM) model with the contextual information regarding dynamics, social neighborhood, and scene semantics to predict future trajectory… 
View on IEEE
doi.org
17 Citations
Highly Influential Citations
1
Background Citations
5
Methods Citations
11

Figures and Tables from this paper

17 Citations

Semantic scene upgrades for trajectory prediction

Experimental results show that utilizing a fully segmented map, for explicit scene semantics, out performs other variants of scene representations (semantic and CNN embedding) for trajectory prediction tasks.

BR-GAN: A Pedestrian Trajectory Prediction Model Combined With Behavior Recognition

This paper builds the Geographical, the Social and the Behavior feature modules based on the GAN framework to process information and presents a trajectory prediction approach, referred to as the BR-GAN, which exploits geographical, social and behavior context-aware.

SRA-LSTM: Social Relationship Attention LSTM for Human Trajectory Prediction

A Social Relationship Attention LSTM (SRA-LSTM) model to predict future trajectories of pedestrian trajectory prediction for surveillance video achieves superior performance compared with state-of-theart methods.

A Novel Graph-Based Trajectory Predictor With Pseudo-Oracle

The Graph-based Trajectory Predictor with Pseudo-Oracle (GTPPO), an encoder–decoder-based method conditioned on pedestrians’ future behaviors, is proposed and evaluated on ETH, UCY, and Stanford Drone datasets, and the results demonstrate state-of-the-art performance.

Social Pooling with Edge Convolutions on Local Connectivity Graphs for Human Trajectory Prediction in Crowded Scenes

A novel multi-layer network architecture based on a new Edge Convolutional operator acting on irregular data which is able to generalize local human-human interactions on a semantic social context is developed and integrated into a state-of-the-art trajectory prediction framework based on Generative Adversarial Networks.

Scene Gated Social Graph: Pedestrian Trajectory Prediction Based on Dynamic Social Graphs and Scene Constraints

In this work, a novel trajectory prediction method named Scene Gated Social Graph (SGSG) is proposed, which uses dynamic graphs to describe the social relationship among pedestrians and achieves superior performance on two widely used trajectory prediction benchmarks.

Intention-Aware Vehicle Trajectory Prediction Based on Spatial-Temporal Dynamic Attention Network for Internet of Vehicles

A driving intention-specific feature fusion mechanism is proposed such that the extracted temporal and social features can be integrated adaptively for the maneuver-based multi-modal trajectory prediction.

TPPO: A Novel Trajectory Predictor with Pseudo Oracle

Results demonstrate that TPPO outperforms state-of-the-art methods with low average and final displacement errors, and the ablation study shows that the prediction performance will not dramatically decrease as sampling times decline during tests.

Review of Pedestrian Trajectory Prediction Methods: Comparing Deep Learning and Knowledge-Based Approaches

A comparison of relatively new deep learning algorithms with classical knowledge-based models that are widely used to simulate pedestrian dynamics shows that the combination of both approaches seems to be promising to overcome disadvantages like the missing explainability of the deep learning approach.

Spatio-Temporal Interaction Aware and Trajectory Distribution Aware Graph Convolution Network for Pedestrian Multimodal Trajectory Prediction

The proposed spatio-temporal interaction aware and trajectory distribution aware graph convolution network and self-attention mechanism can predict socially acceptable future trajectories and address the problem of the propagation and accumulation of prediction errors.

References

SHOWING 1-10 OF 15 REFERENCES

SS-LSTM: A Hierarchical LSTM Model for Pedestrian Trajectory Prediction

A novel hierarchical LSTM-based network is proposed to consider both the influence of social neighbourhood and scene layouts in pedestrian trajectory prediction, showing that the method outperforms other methods and that using circular shape neighbourhood improves the prediction accuracy.

Social LSTM: Human Trajectory Prediction in Crowded Spaces

This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.

Convolutional Social Pooling for Vehicle Trajectory Prediction

  • Nachiket DeoM. Trivedi
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2018
This paper proposes an LSTM encoder-decoder model that uses convolutional social pooling as an improvement to social Pooling layers for robustly learning interdependencies in vehicle motion and outputs a multi-modal predictive distribution over future trajectories based on maneuver classes.

Context-Aware Trajectory Prediction

This work proposes a “context-aware” recurrent neural network LSTM model, which can learn and predict human motion in crowded spaces such as a sidewalk, a museum or a shopping mall, and evaluates the model on a public pedestrian datasets.

An Evaluation of Trajectory Prediction Approaches and Notes on the TrajNet Benchmark

It is shown that a Recurrent-Encoder with a Dense layer stacked on top, referred to as RED-predictor, is able to achieve sophisticated results compared to elaborated models in such scenarios and some recommendations for overcoming demonstrated shortcomings are given.

Fully convolutional networks for semantic segmentation

The key insight is to build “fully convolutional” networks that take input of arbitrary size and produce correspondingly-sized output with efficient inference and learning.

SegNet: A Deep Convolutional Encoder-Decoder Architecture for Robust Semantic Pixel-Wise Labelling

The results show that SegNet achieves state-of-the-art performance even without use of additional cues such as depth, video frames or post-processing with CRF models.

Semantic object classes in video: A high-definition ground truth database

Probabilistic vehicle trajectory prediction over occupancy grid map via recurrent neural network

The proposed trajectory prediction method employs the recurrent neural network called long short-term memory (LSTM) to analyze the temporal behavior and predict the future coordinate of the surrounding vehicles.

SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints

SoPhie is presented; an interpretable framework based on Generative Adversarial Network (GAN), which leverages two sources of information, the path history of all the agents in a scene, and the scene context information, using images of the scene.