Learning to Simulate Realistic LiDARs

@article{Guillard2022LearningTS,
  title={Learning to Simulate Realistic LiDARs},
  author={Benoit Guillard and Sai Vemprala and Jayesh K. Gupta and Ondřej Mik{\vs}{\'i}k and Vibhav Vineet and P. Fua and Ashish Kapoor},
  journal={2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
  year={2022},
  pages={8173-8180}
}
  • Benoit GuillardSai Vemprala Ashish Kapoor
  • Published 22 September 2022
  • Computer Science, Environmental Science
  • 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
Simulating realistic sensors is a challenging part in data generation for autonomous systems, often involving carefully handcrafted sensor design, scene properties, and physics modeling. To alleviate this, we introduce a pipeline for data-driven simulation of a realistic LiDAR sensor. We propose a model that learns a mapping between RGB images and corresponding LiDAR features such as raydrop or perpoint intensities directly from real datasets. We show that our model can learn to encode… 
View on IEEE
arxiv.org

Figures and Tables from this paper

References

SHOWING 1-10 OF 37 REFERENCES

LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World

This work develops a novel simulator that captures both the power of physics-based and learning-based simulation, and showcases LiDARsim's usefulness for perception algorithms-testing on long-tail events and end-to-end closed-loop evaluation on safety-critical scenarios.

Unsupervised Neural Sensor Models for Synthetic LiDAR Data Augmentation

This work proposes two unsupervised neural sensor models based on unpaired domain translations with CycleGANs and Neural Style Transfer techniques and provides a framework for data augmentation and evaluation of the developed sensor models, through extrinsic object detection task evaluation.

Learning to Predict Lidar Intensities

We propose a data-driven method for simulating lidar sensors. The method reads computer-generated data, and (i) extracts geometrically simulated lidar point clouds and (ii) predicts the strength of…

LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving

This work forms the problem as image translation from unpaired data and employs CycleGANs to solve the sensor modeling problem for LiDAR, to produce realisticLiDAR from simulated LiD AR (sim2real) and generates high-resolution, realistic LiDar from lower resolution one (real2real).

Learning to Drop Points for LiDAR Scan Synthesis

  • Kazuto NakashimaR. Kurazume
  • Computer Science, Environmental Science
    2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2021
This paper proposes a novel generative model for learning LiDAR data based on generative adversarial networks and introduces measurement uncertainty into the generation process, which allows the model to learn a disentangled representation of the underlying shape and the dropout noises from a collection of real LiDar data.

Simulated Intensity Rendering of 3D LiDAR using Generative Adversarial Network

  • Seung-chan MokGon-woo Kim
  • Computer Science, Environmental Science
    2021 IEEE International Conference on Big Data and Smart Computing (BigComp)
  • 2021
This paper proposes realistic intensity rendering method with simulator by generative adversarial network (GAN) trained with unannotated real data and develops LiDAR projection preprocessing method for operating convolution networks.

Augmented LiDAR Simulator for Autonomous Driving

This letter proposes a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., vehicles, pedestrians, and other movable objects) and describes the placement of obstacles that is critical for performance enhancement.

Deep Generative Modeling of LiDAR Data

This work shows that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map, and proposes a novel data representation that augments the 2D signal with absolute positional information.

End-to-end sensor modeling for LiDAR Point Cloud

This work proposes a novel Deep Learning-based LiDAR sensor model that models the sensor echos, using a Deep Neural Network to model echo pulse widths learned from real data using Polar Grid Maps (PGM).

RangeNet ++: Fast and Accurate LiDAR Semantic Segmentation

This paper proposes a novel post-processing algorithm that deals with problems arising from this intermediate representation of range images as an intermediate representation in combination with a Convolutional Neural Network exploiting the rotating LiDAR sensor model.