3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data

  title={3DOF Pedestrian Trajectory Prediction Learned from Long-Term Autonomous Mobile Robot Deployment Data},
  author={Li Sun and Zhi Yan and Sergi Molina Mellado and Marc Hanheide and Tom Duckett},
  journal={2018 IEEE International Conference on Robotics and Automation (ICRA)},
  • Li SunZhi Yan T. Duckett
  • Published 30 September 2017
  • Computer Science
  • 2018 IEEE International Conference on Robotics and Automation (ICRA)
This paper presents a novel 3DOF pedestrian trajectory prediction approach for autonomous mobile service robots. While most previously reported methods are based on learning of 2D positions in monocular camera images, our approach uses range-finder sensors to learn and predict 3DOF pose trajectories (i.e. 2D position plus 1D rotation within the world coordinate system). Our approach, T-Pose-LSTM (Temporal 3DOF-Pose Long-Short-Term Memory), is trained using long-term data from real-world robot… 

Figures and Tables from this paper

Effective Interaction-aware Trajectory Prediction using Temporal Convolutional Neural Networks

This work proposes the IA-TCNN architecture to address the problem of efficient trajectory prediction for multiple interacting pedestrians in a scene, which leverages the previous motion information of all pedestrians in order to aggregate trajectories of the most relevant pedestrians within the scene, thus predicting an accurate future trajectory.

A Two-Block RNN-Based Trajectory Prediction From Incomplete Trajectory

This paper introduces a two-block RNN model that approximates the inference steps of the Bayesian filtering framework and seeks the optimal estimation of the hidden state when miss-detection occurs, and uses two RNNs depending on the detection result.

Pedestrian Trajectory Prediction in Extremely Crowded Scenarios

A novel trajectory prediction model is proposed to capture the relative motion of pedestrians in extremely crowded scenarios and is stable for predictions of varying length even when the model is trained with a controlled short trajectory sequence.

Exploring Map-based Features for Efficient Attention-based Vehicle Motion Prediction

This work explores how to achieve competitive performance on the Argoverse 1.0 Benchmark using efficient attention-based models, which take as input the past trajectories and map-based features from minimal map information to ensure efficient and reliable MP.

Multisensor Online Transfer Learning for 3D LiDAR-Based Human Detection with a Mobile Robot

A framework allowing a robot to learn a new 3D LiDAR-based human classifier from other sensors over time, taking advantage of a multisensor tracking system, implemented and tested on a real-world dataset collected by a mobile robot.

Location-Velocity Attention for Pedestrian Trajectory Prediction

A module is designed to tweak the LSTM network and an attention mechanism is trained to learn to optimally combine the location and the velocity information of pedestrians in the prediction process to predict trajectories.

A Location-Velocity-Temporal Attention LSTM Model for Pedestrian Trajectory Prediction

Evaluation on using LVTA to generate trajectories of different prediction lengths and on new scenes without the need of retraining confirms that it has good generalizability.

Multisensor Online Transfer Learning for 3D LiDAR-based Human Classification with a Mobile Robot

A framework allowing a robot to learn a new 3D LiDAR-based human classifier from other sensors over time, taking advantage of a multisensor tracking system, and is implemented and tested on a real-world dataset collected by a mobile robot.

Context Dependent Trajectory Generation using Sequence-to-Sequence Models for Robotic Toilet Cleaning

A Sequence-to-Sequence (Seq2Seq) based neural network model is proposed to generate robot trajectories in configuration space given a context variable based on real-world measurements in Cartesian space using the offset between a target pose and the actual pose after localization as the context variable.

A Review of Deep Learning-Based Methods for Pedestrian Trajectory Prediction

The current paper reviews the most recent deep learning-based solutions for the problem of pedestrian trajectory prediction along with employed sensors and afferent processing methodologies, and it performs an overview of the available datasets, performance metrics used in the evaluation process, and practical applications.



Real-time multisensor people tracking for human-robot spatial interaction

A fully integrated people perception framework, designed to run in real-time on a mobile robot, and fully implemented into the Robot Operating System (ROS) in a small proof of concept experiment.

Social LSTM: Human Trajectory Prediction in Crowded Spaces

This work proposes an LSTM model which can learn general human movement and predict their future trajectories and outperforms state-of-the-art methods on some of these datasets.

Context-Aware Trajectory Prediction

This work proposes a “context-aware” recurrent neural network LSTM model, which can learn and predict human motion in crowded spaces such as a sidewalk, a museum or a shopping mall, and evaluates the model on a public pedestrian datasets.

FreMEn: Frequency Map Enhancement for Long-Term Mobile Robot Autonomy in Changing Environments

Transformation of the spectral model to the time domain allows for the prediction of the future environment states, which improves the robot's long-term performance in changing environments.

On multi-modal people tracking from mobile platforms in very crowded and dynamic environments

A fully integrated real-time multi-modal laser/RGB-D people tracking framework for moving platforms in environments like a busy airport terminal is proposed, which indicates that more complex data association methods may not always be the better choice, and derive possible future research directions.

Real-time RGB-D based people detection and tracking for mobile robots and head-worn cameras

This approach combines RGB-D visual odometry estimation, region-of-interest processing, ground plane estimation, pedestrian detection, and multi-hypothesis tracking components into a robust vision system that runs at more than 20fps on a laptop.

Multi-model hypothesis tracking of groups of people in RGB-D data

  • Timm LinderK. Arras
  • Computer Science
    17th International Conference on Information Fusion (FUSION)
  • 2014
This paper addresses the problem of detecting groups of people from learned social relations between individuals with the goal to reliably track group formation processes, and tracks and reason about multiple social grouping hypotheses in a recursive way.

You'll never walk alone: Modeling social behavior for multi-target tracking

A model of dynamic social behavior, inspired by models developed for crowd simulation, is introduced, trained with videos recorded from birds-eye view at busy locations, and applied as a motion model for multi-people tracking from a vehicle-mounted camera.

Unfreezing the robot: Navigation in dense, interacting crowds

IGP is developed, a nonparametric statistical model based on dependent output Gaussian processes that can estimate crowd interaction from data that naturally captures the non-Markov nature of agent trajectories, as well as their goal-driven navigation.

Learning Social Etiquette: Human Trajectory Understanding In Crowded Scenes

This paper contributes a new large-scale dataset that collects videos of various types of targets that navigate in a real world outdoor environment such as a university campus and introduces a new characterization that describes the “social sensitivity” at which two targets interact.