Learning to Generate Realistic LiDAR Point Clouds

  title={Learning to Generate Realistic LiDAR Point Clouds},
  author={Vlas Zyrianov and Xiyue Zhu and Shenlong Wang},
. We present LiDARGen, a novel, effective, and controllable generative model that produces realistic LiDAR point cloud sensory readings. Our method leverages the powerful score-matching energy-based model and formulates the point cloud generation process as a stochastic denoising process in the equirectangular view. This model allows us to sample diverse and high-quality point cloud samples with guaranteed physical feasibility and controllability. We validate the effectiveness of our method on… 

Figures from this paper



Learning to Drop Points for LiDAR Scan Synthesis

  • Kazuto NakashimaR. Kurazume
  • Computer Science, Environmental Science
    2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2021
This paper proposes a novel generative model for learning LiDAR data based on generative adversarial networks and introduces measurement uncertainty into the generation process, which allows the model to learn a disentangled representation of the underlying shape and the dropout noises from a collection of real LiDar data.

LiDARsim: Realistic LiDAR Simulation by Leveraging the Real World

This work develops a novel simulator that captures both the power of physics-based and learning-based simulation, and showcases LiDARsim's usefulness for perception algorithms-testing on long-tail events and end-to-end closed-loop evaluation on safety-critical scenarios.

PointFlow: 3D Point Cloud Generation With Continuous Normalizing Flows

A principled probabilistic framework to generate 3D point clouds by modeling them as a distribution of distributions with the invertibility of normalizing flows enables the computation of the likelihood during training and allows the model to train in the variational inference framework.

SynLiDAR: Learning From Synthetic LiDAR Sequential Point Cloud for Semantic Segmentation

Extensive experiments over multiple data augmentation and semi-supervised semantic segmentation tasks show very positive outcome including SynLiDAR can either train better models or reduce real-world annotated data without sacrificing performance, and PCT-Net translated data further improve model performance consistently.

Deep Generative Modeling of LiDAR Data

This work shows that one can adapt deep generative models for this task by unravelling lidar scans into a 2D point map, and proposes a novel data representation that augments the 2D signal with absolute positional information.

LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving

This work forms the problem as image translation from unpaired data and employs CycleGANs to solve the sensor modeling problem for LiDAR, to produce realisticLiDAR from simulated LiD AR (sim2real) and generates high-resolution, realistic LiDar from lower resolution one (real2real).

Augmented LiDAR Simulator for Autonomous Driving

This letter proposes a novel LiDAR simulator that augments real point cloud with synthetic obstacles (e.g., vehicles, pedestrians, and other movable objects) and describes the placement of obstacles that is critical for performance enhancement.

Learning Representations and Generative Models for 3D Point Clouds

A deep AutoEncoder network with state-of-the-art reconstruction quality and generalization ability is introduced with results that outperform existing methods on 3D recognition tasks and enable shape editing via simple algebraic manipulations.

LiDAR Point Cloud Generation for SLAM Algorithm Evaluation

A LiDAR simulator that delivers accurate 3D point clouds in real time that is compatible with the Robotic Operating System (ROS) and can be used interchangeably with data from actual sensors, which enables easy testing, SLAM algorithm parameter tuning and deployment.

Diffusion Probabilistic Models for 3D Point Cloud Generation

  • Shitong LuoWei Hu
  • Computer Science
    2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2021
A probabilistic model for point cloud generation is presented, which is fundamental for various 3D vision tasks such as shape completion, upsampling, synthesis and data augmentation, and derives the variational bound in closed form for training and provides implementations of the model.