End-to-end Contextual Perception and Prediction with Interaction Transformer

  title={End-to-end Contextual Perception and Prediction with Interaction Transformer},
  author={L. Li and Bin Yang and Ming Liang and Wenyuan Zeng and Mengye Ren and Sean Segal and R. Urtasun},
  journal={2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  • L. Li, Bin Yang, +4 authors R. Urtasun
  • Published 2020
  • Computer Science
  • 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • In this paper, we tackle the problem of detecting objects in 3D and forecasting their future motion in the context of self-driving. Towards this goal, we design a novel approach that explicitly takes into account the interactions between actors. To capture their spatial-temporal dependencies, we propose a recurrent neural network with a novel Transformer [1] architecture, which we call the Interaction Transformer. Importantly, our model can be trained end-to-end, and runs in real-time. We… CONTINUE READING
    12 Citations

    Figures, Tables, and Topics from this paper

    DSDNet: Deep Structured self-Driving Network
    • 11
    • PDF
    Universal Embeddings for Spatio-Temporal Tagging of Self-Driving Logs
    • 1
    • PDF
    Social NCE: Contrastive Learning of Socially-aware Motion Representations
    • PDF
    LaneRCNN: Distributed Representations for Graph-Centric Motion Forecasting
    • PDF
    Deep Structured Reactive Planning
    • PDF
    Implicit Latent Variable Model for Scene-Consistent Motion Forecasting
    • 17
    • PDF
    Learning Lane Graph Representations for Motion Forecasting
    • 12
    • PDF
    V2VNet: Vehicle-to-Vehicle Communication for Joint Perception and Prediction
    • 7
    • PDF


    CAR-Net: Clairvoyant Attentive Recurrent Network
    • 70
    • PDF
    SpAGNN: Spatially-Aware Graph Neural Networks for Relational Behavior Forecasting from Sensor Data
    • 47
    • PDF
    Social LSTM: Human Trajectory Prediction in Crowded Spaces
    • 1,048
    • PDF
    SoPhie: An Attentive GAN for Predicting Paths Compliant to Social and Physical Constraints
    • 249
    • PDF
    STGAT: Modeling Spatial-Temporal Interactions for Human Trajectory Prediction
    • 57
    • PDF
    Uncertainty-aware Short-term Motion Prediction of Traffic Actors for Autonomous Driving
    • 55
    • PDF
    End-To-End Interpretable Neural Motion Planner
    • 77
    • PDF
    Convolutional Social Pooling for Vehicle Trajectory Prediction
    • N. Deo, M. Trivedi
    • Computer Science
    • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
    • 2018
    • 177
    • PDF
    Fast and Furious: Real Time End-to-End 3D Detection, Tracking and Motion Forecasting with a Single Convolutional Net
    • 236
    • PDF