Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View

  title={Cycle and Semantic Consistent Adversarial Domain Adaptation for Reducing Simulation-to-Real Domain Shift in LiDAR Bird's Eye View},
  author={Alejandro Barrera and Jorge Beltr'an and Carlos Guindel and Jose Iglesias and Fernando Abell'an Garc'ia},
  journal={2021 IEEE International Intelligent Transportation Systems Conference (ITSC)},
The performance of object detection methods based on LiDAR information is heavily impacted by the availability of training data, usually limited to certain laser devices. As a result, the use of synthetic data is becoming popular when training neural network models, as both sensor specifications and driving scenarios can be generated ad-hoc. However, bridging the gap between virtual and real environments is still an open challenge, as current simulators cannot completely mimic real LiDAR… 
2 Citations

Figures and Tables from this paper

DALi: Domain Adaptation in LiDAR Point Clouds for 3D Obstacle Detection

An unsupervised domain adaptation training strategy is presented, so that features encoded by PointNet-like backbones are agnostic to the input domain, and results on the KITTI and nuScenes benchmarks show a notable improvement over the baseline.

Learning to Simulate Realistic LiDARs

A model that learns a mapping between RGB images and corresponding LiDAR features such as raydrop or per- point intensities directly from real datasets is proposed, and it is shown that this model can learn to encode realistic effects such as dropped points on transparent surfaces or high intensity returns on reflective materials.



Domain Adaptation for Vehicle Detection from Bird's Eye View LiDAR Point Cloud Data

The proposed framework is based on the deep cycle-consistent generative adversarial networks (CycleGAN) architecture and has shown competitive results with an improvement of more than 7% in average precision score over other baseline approaches when tested on real BEV point cloud images.

SqueezeSegV2: Improved Model Structure and Unsupervised Domain Adaptation for Road-Object Segmentation from a LiDAR Point Cloud

This work introduces a new model SqueezeSegV2, which is more robust against dropout noises in LiDAR point cloud and therefore achieves significant accuracy improvement, and a domain-adaptation training pipeline consisting of three major components: learned intensity rendering, geodesic correlation alignment, and progressive domain calibration.

ePointDA: An End-to-End Simulation-to-Real Domain Adaptation Framework for LiDAR Point Cloud Segmentation

ePointDA is proposed, a novel end-to-end framework that enables ePointDA to bridge the domain shift at the pixel-level by explicitly rendering dropout noise for synthetic LiDAR and at the feature- level by spatially aligning the features between different domains, without requiring the real-world statistics.

LiDAR Sensor modeling and Data augmentation with GANs for Autonomous driving

This work forms the problem as image translation from unpaired data and employs CycleGANs to solve the sensor modeling problem for LiDAR, to produce realisticLiDAR from simulated LiD AR (sim2real) and generates high-resolution, realistic LiDar from lower resolution one (real2real).

Train in Germany, Test in the USA: Making 3D Object Detectors Generalize

This paper provides a first baseline for 3D object detection adaptation across countries, and gives hope that the underlying problem may be more within grasp than one may have hoped to believe.

A Method for Synthetic LiDAR Generation to Create Annotated Datasets for Autonomous Vehicles Perception

This work presents a method to generate new point clouds datasets as captured by a real LiDAR device, using a proposed pipeline that makes use of multiple frames to perform an accurate 3D reconstruction of the scene in the spherical coordinates system.

LiDARNet: A Boundary-Aware Domain Adaptation Model for Lidar Point Cloud Semantic Segmentation

A boundary-aware domain adaptation model for Lidar point cloud semantic segmentation that is capable of achieving the state of art performance on the source domain and get 44.0\% mIoU on the target domain dataset.

Point-Based Multilevel Domain Adaptation for Point Cloud Segmentation

A 3D point-based UDA method for point cloud semantic segmentation that introduces point- and set-level domain adaptive modules to achieve feature alignment between the domains and evaluates the effectiveness of this method with two experiments.

BEV-Seg: Bird's Eye View Semantic Segmentation Using Geometry and Semantic Point Cloud

A novel 2-staged perception pipeline explicitly predicts pixel depths and combines them with pixel semantics in an efficient manner, allowing the model to leverage depth information to infer objects' spatial locations in the BEV.

SalsaNext: Fast, Uncertainty-Aware Semantic Segmentation of LiDAR Point Clouds

This paper introduces SalsaNext for the uncertainty-aware semantic segmentation of a full 3D LiDAR point cloud in real-time and provides a thorough quantitative evaluation on the Semantic-KITTI dataset, which demonstrates that the proposed Salsa next outperforms other state-of-the-art semantic segmentations networks and ranks first on theSemantic- KITTI leaderboard.