• Corpus ID: 237258010

Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR

  title={Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR},
  author={Ziyue Feng and Longlong Jing and Peng Yin and Yingli Tian and Bing Li},
: Self-supervised monocular depth prediction provides a cost-effective solution to obtain the 3D location of each pixel. However, the existing approaches usually lead to unsatisfactory accuracy, which is critical for autonomous robots. In this paper, we propose FusionDepth, a novel two-stage network to advance the self-supervised monocular dense depth learning by leveraging low-cost sparse (e.g. 4-beam) LiDAR. Unlike the existing methods that use sparse LiDAR mainly in a manner of time… 

Figures and Tables from this paper

Deep Depth Completion from Extremely Sparse Data: A Survey

A comprehensive literature review of the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods is provided.

CamLiFlow: Bidirectional Camera-LiDAR Fusion for Joint Optical Flow and Scene Flow Estimation

This paper proposes a novel end-to-end framework, called CamLiFlow, which consists of 2D and 3D branches with multiple bidirectional connections between them in speci fi c layers and applies a point-based 3D branch to better extract the geometric features and design a symmetric learnable operator to fuse dense image features and sparse point features.

Revisiting PatchMatch Multi-View Stereo for Urban 3D Reconstruction

A complete pipeline for image-based 3D reconstruction of urban scenarios is proposed, based on PatchMatch Multi-View Stereo, with state of the art performances against both classical MVS algorithms and monocular depth networks on the KITTI dataset.

A Comprehensive Survey of Depth Completion Approaches

A novel taxonomy of depth completion approaches is presented, different state-of-the-art techniques within each category for depth completion of LiDAR data are reviewed, and quantitative results for the approaches on KITTI and NYUv2 depth completion benchmark datasets are provided.

Deep Depth Completion: A Survey

A comprehensive literature review of the related studies from the design aspects of network architectures, loss functions, benchmark datasets, and learning strategies with a proposal of a novel taxonomy that categorizes existing methods.



Self-Supervised Sparse-to-Dense: Self-Supervised Depth Completion from LiDAR and Monocular Camera

A deep regression model is developed to learn a direct mapping from sparse depth (and color images) input to dense depth prediction and a self-supervised training framework that requires only sequences of color and sparse depth images, without the need for dense depth labels is proposed.

Pseudo-LiDAR++: Accurate Depth for 3D Object Detection in Autonomous Driving

This paper provides substantial advances to the pseudo-LiDAR framework through improvements in stereo depth estimation, and proposes a depth-propagation algorithm, guided by the initial depth estimates, to diffuse these few exact measurements across the entire depth map.

Semi-Supervised Deep Learning for Monocular Depth Map Prediction

This paper proposes a novel approach to depth map prediction from monocular images that learns in a semi-supervised way and uses sparse ground-truth depth for supervised learning, and also enforces the deep network to produce photoconsistent dense depth maps in a stereo setup using a direct image alignment loss.

Robust Semi-Supervised Monocular Depth Estimation with Reprojected Distances

This paper proposes a novel supervised loss term that complements the widely used photometric loss, and shows how it can be used to train robust semi-supervised monocular depth estimation models.

Pseudo-LiDAR From Visual Depth Estimation: Bridging the Gap in 3D Object Detection for Autonomous Driving

This paper proposes to convert image-based depth maps to pseudo-LiDAR representations --- essentially mimicking the LiDAR signal, and achieves impressive improvements over the existing state-of-the-art in image- based performance.

Digging Into Self-Supervised Monocular Depth Estimation

It is shown that a surprisingly simple model, and associated design choices, lead to superior predictions, and together result in both quantitatively and qualitatively improved depth maps compared to competing self-supervised methods.

Depth Completion From Sparse LiDAR Data With Depth-Normal Constraints

A unified CNN framework is proposed that models the geometric constraints between depth and surface normal in a diffusion module and predicts the confidence of sparse LiDAR measurements to mitigate the impact of noise.

Deep Learning based Monocular Depth Prediction: Datasets, Methods and Applications

This survey introduces the datasets for depth estimation, and gives a comprehensive introduction of the methods from three perspectives: supervised learning-based methods, unsupervised learning- based methods, and sparse samples guidance-based Methods.

Balanced Depth Completion between Dense Depth Inference and Sparse Range Measurements via KISS-GP

  • Sungho YoonAyoung Kim
  • Computer Science
    2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
  • 2020
The fusion of these two modalities as a depth completion (DC) problem by dividing the role of depth inference and depth regression is introduced and the accuracy and robustness of the method outperform state-of-the-art unsupervised methods for sparse and biased measurements.

End-to-End Pseudo-LiDAR for Image-Based 3D Object Detection

A new framework based on differentiable Change of Representation (CoR) modules that allow the entire PL pipeline to be trained end-to-end and is compatible with most state-of-the-art networks for both tasks and in combination with PointRCNN improves over PL consistently across all benchmarks.