Efficient and Robust LiDAR-Based End-to-End Navigation

  title={Efficient and Robust LiDAR-Based End-to-End Navigation},
  author={Zhijian Liu and Alexander Amini and Sibo Zhu and Sertaç Karaman and Song Han and Daniela Rus},
  journal={2021 IEEE International Conference on Robotics and Automation (ICRA)},
Deep learning has been used to demonstrate end-to-end neural network learning for autonomous vehicle control from raw sensory input. While LiDAR sensors provide reliably accurate information, existing end-to-end driving solutions are mainly based on cameras since processing 3D data requires a large memory footprint and computation cost. On the other hand, increasing the robustness of these systems is also critical; however, even estimating the model’s uncertainty is very challenging due to the… 

Figures from this paper

Robust Monocular Localization in Sparse HD Maps Leveraging Multi-Task Uncertainty Estimation

An efficient multi-task uncertainty-aware perception module is proposed, which covers semantic segmentation, as well as bounding box detection, to enable the localization of vehicles in sparse maps, containing only lane borders and traffic lights.

A Robust Sidewalk Navigation Method for Mobile Robots Based on Sparse Semantic Point Cloud

The results show that this method enables the robot to navigate on the sidewalk robustly during day and night, and the LSTM neural network is explored to effectively leverage the historical context and derive correct decisions.

A Survey on Evidential Deep Learning For Single-Pass Uncertainty Estimation

This survey aims to familiarize the reader with an alternative class of models based on the concept of Evidential Deep Learning, which allow uncertainty estimation in a single model and forward pass by parameterizing distributions over distributions.

Enable Deep Learning on Mobile Devices: Methods, Systems, and Applications

This article provides an overview of efficient deep learning methods, systems, and applications by introducing popular model compression methods, including pruning, factorization, quantization, as well as compact model design.

MIDGARD: A Simulation Platform for Autonomous Navigation in Unstructured Environments

MIDGARD differs from other major simulation platforms in that it proposes a highly configurable procedural landscape generation pipeline, which enables autonomous agents to be trained in diverse scenarios while reducing the efforts and costs needed to create digital content from scratch.

FlatFormer: Flattened Window Attention for Efficient Point Cloud Transformer

FlatFormer is the first point cloud transformer that achieves real-time performance on edge GPUs and is faster than sparse convolutional methods while achieving on-par or even superior accuracy on large-scale benchmarks.

The Effects of Fully Connected Layers Adjustment for Lightweight Convolutional Neural Networks

This work is proving how and why the number of nodes in fully connected (dense) layers is the most important factor for reducing the size of a CNN.

The Unreasonable Effectiveness of Deep Evidential Regression

The theoretical shortcomings are detailed and the performance on synthetic and real-world data sets are analyzed, show-ing that Deep Evidential Regression is a heuristic rather than an exact uncertainty quantification.

Single chip photonic deep neural network with accelerated training

This work experimentally demonstrates a fully-integrated coherent optical neural network (FICONN) architecture for a 3-layer DNN comprising 12 NOFUs and three CMXUs operating in the telecom C-band, and opens the path to inference at nanosecond latency and femtojoule per operation energy efficiency.

VISTA 2.0: An Open, Data-driven Simulator for Multimodal Sensing and Policy Learning for Autonomous Vehicles

The ability to train and test perception-to-control policies across each of the sensor types and showcase the power of this approach via deployment on a full scale autonomous vehicle are demonstrated.



Variational End-to-End Navigation and Localization

This paper defines a novel variational network capable of learning from raw camera data of the environment as well as higher level roadmaps to predict a full probability distribution over the possible control commands and formulate how this model can be used to localize the robot according to correspondences between the map and the observed visual road topology.

Multimodal End-to-End Autonomous Driving

This paper analyses whether combining RGB and depth modalities, i.e. using RGBD data, produces better end-to-end AI drivers than relying on a single modality and shows how, indeed, early fusion multimodality outperforms single-modality.

Sensor modality fusion with CNNs for UGV autonomous driving in indoor environments

A novel end-to-end learning framework to enable ground vehicles to autonomously navigate unknown environments by fusing raw pixels from cameras and depth measurements from a LiDAR provides a potential to navigate around static and dynamic obstacles and to handle changes in environment geometry without being trained for these tasks.

End to End Learning for Self-Driving Cars

A convolutional neural network is trained to map raw pixels from a single front-facing camera directly to steering commands and it is argued that this will eventually lead to better performance and smaller systems.

Visual localization within LIDAR maps for automated urban driving

  • R. W. WolcottR. Eustice
  • Computer Science
    2014 IEEE/RSJ International Conference on Intelligent Robots and Systems
  • 2014
This paper proposes to localize a single monocular camera within a 3D prior ground-map, generated by a survey vehicle equipped with 3D LIDAR scanners, to obtain comparable localization accuracy with significantly cheaper, commodity cameras.

Evaluating Uncertainty Quantification in End-to-End Autonomous Driving Control

It is demonstrated how quantitative measures of uncertainty can be extracted in real-time, and their quality evaluated in end-to-end controllers for self-driving cars and found that mutual information, a measure of uncertainty in classification networks, is a promising indicator of forthcoming crashes.

Map-Based Precision Vehicle Localization in Urban Environments

This work proposes a technique for high-accuracy localization of moving vehicles that utilizes maps of urban environments that integrates GPS, IMU, wheel odometry, and LIDAR data acquired by an instrumented vehicle, to generate high-resolution environment maps.

Searching Efficient 3D Architectures with Sparse Point-Voxel Convolution

This work proposes Sparse Point-Voxel Convolution (SPVConv), a lightweight 3D module that equips the vanilla Sparse Convolution with the high-resolution point-based branch, and presents 3D Neural Architecture Search (3D-NAS) to search the optimal network architecture over this diverse design space efficiently and effectively.

Robust vehicle localization in urban environments using probabilistic maps

  • Jesse LevinsonS. Thrun
  • Computer Science, Environmental Science
    2010 IEEE International Conference on Robotics and Automation
  • 2010
This work proposes an extension to this approach to vehicle localization that yields substantial improvements over previous work in vehicle localization, including higher precision, the ability to learn and improve maps over time, and increased robustness to environment changes and dynamic obstacles.

End-to-End Driving Via Conditional Imitation Learning

This work evaluates different architectures for conditional imitation learning in vision-based driving and conducts experiments in realistic three-dimensional simulations of urban driving and on a 1/5 scale robotic truck that is trained to drive in a residential area.