Learning Depth with Convolutional Spatial Propagation Network

@article{Cheng2020LearningDW,
  title={Learning Depth with Convolutional Spatial Propagation Network},
  author={Xinjing Cheng and Peng Wang and Ruigang Yang},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  volume={42},
  pages={2361-2379}
}
Depth prediction is one of the fundamental problems in computer vision. [...] Key Method In practice, we further extend CSPN in two aspects: 1) take a sparse depth map as additional input, which is useful for the task of sparse to dense (a.k.a depth completion); 2) we propose 3D CSPN to handle features with one additional dimension, which is effective in the task of stereo matching using 3D cost volume. For the tasks of depth completion, we experimented the proposed CPSN conjunct algorithms over NYU v2 and…Expand
CSPN++: Learning Context and Resource Aware Convolutional Spatial Propagation Networks for Depth Completion
TLDR
This paper proposes CSPN++, which further improves its effectiveness and efficiency by learning adaptive convolutional kernel sizes and the number of iterations for the propagation, thus the context and computational resources needed at each pixel could be dynamically assigned upon requests.
Dense-CNN: Dense convolutional neural network for stereo matching using multiscale feature connection
TLDR
A dense convolutional neural network-based stereo matching method with multiscale feature connection, named Dense-CNN, which achieves superior performance on computational accuracy and robustness of disparity estimation, especially achieving the significant benefit of feature preservation in ill-posed regions.
Superpixel Segmentation With Fully Convolutional Networks
TLDR
A novel method that employs a simple fully convolutional network to predict superpixels on a regular image grid and develops a downsampling/upsampling scheme for deep networks with the goal of generating high-resolution outputs for dense prediction tasks.
Learning Guided Convolutional Network for Depth Completion
TLDR
Inspired by the guided image filtering, a novel guided network is designed to predict kernel weights from the guidance image and these predicted kernels are then applied to extract the depth image features.
Belief Propagation Reloaded: Learning BP-Layers for Labeling Problems
TLDR
One of the simplest inference methods is taken, a truncated max-product Belief Propagation, and added what is necessary to make it a proper component of a deep learning model: connect it to learning formulations with losses on marginals and compute the backprop operation.
Multi-Dimensional Residual Dense Attention Network for Stereo Matching
TLDR
An end-to-end multi-dimensional residual dense attention network (MRDA-Net) is proposed, focusing on more comprehensive pixel-wise feature extraction, and achieves the state-of-the-art accuracy on Scene Flow dataset and KITTI 2012 and KittI 2015 Stereo datasets.
Learning Structure Affinity for Video Depth Estimation
TLDR
This paper proposes a convolutional spatial temporal propagation network (CSTPN) that learns affinity among neighbouring video frames and employs a structure knowledge distillation scheme that transfers the spatial temporal affinity learned by cumbersome network to compact network.
Light-weight network for real-time adaptive stereo depth estimation
TLDR
A light-weight adaptive network (LWANet) is proposed by combining the self-supervised learning method to perform online adaptive stereo depth estimation for low computation cost and low GPU memory space.
RigNet: Repetitive Image Guided Network for Depth Completion
TLDR
This work explores a repetitive design in the authors' image guided network to sufficiently and gradually recover depth values and proposes an adaptive fusion mechanism to effectively aggregate multi-step depth features.
Dedge-AGMNet: A Robust Multi-task Learning Network for Stereo Matching and Depth Edge Detection
TLDR
The depth edge cues and multi-scale context information are both beneficial to explore potential corresponding points in ill-posed regions and the Dedge-AGMNet network outperforms other stereo matching networks and shows very strong robustness for different environments.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 111 REFERENCES
Spatial Pyramid Pooling in Deep Convolutional Networks for Visual Recognition
TLDR
This work equips the networks with another pooling strategy, “spatial pyramid pooling”, to eliminate the above requirement, and develops a new network structure, called SPP-net, which can generate a fixed-length representation regardless of image size/scale.
Deeper Depth Prediction with Fully Convolutional Residual Networks
TLDR
A fully convolutional architecture, encompassing residual learning, to model the ambiguous mapping between monocular images and depth maps is proposed and a novel way to efficiently learn feature map up-sampling within the network is presented.
Conditional Random Fields as Recurrent Neural Networks
TLDR
A new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling is introduced, and top results are obtained on the challenging Pascal VOC 2012 segmentation benchmark.
Structured Attention Guided Convolutional Neural Fields for Monocular Depth Estimation
TLDR
This paper introduces a novel approach for monocular depth estimation which is competitive with previous methods on the KITTI benchmark and outperforms the state of the art on the NYU Depth V2 dataset.
Learning Affinity via Spatial Propagation Networks
TLDR
Experiments on the HELEN face parsing and PASCAL VOC-2012 semantic segmentation tasks show that the spatial propagation network provides a general, effective and efficient solution for generating high-quality segmentation results.
Deep convolutional neural fields for depth estimation from a single image
TLDR
A deep structured learning scheme which learns the unary and pairwise potentials of continuous CRF in a unified deep CNN framework and can be used for depth estimations of general scenes with no geometric priors nor any extra information injected.
Multi-scale Continuous CRFs as Sequential Deep Networks for Monocular Depth Estimation
TLDR
This paper addresses the problem of depth estimation from a single still image by designing a novel CNN implementation of mean-field updates for continuous CRFs and demonstrates the effectiveness of the proposed approach and establishes new state of the art results on publicly available datasets.
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
SGM-Nets: Semi-Global Matching with Neural Networks
  • A. Seki, M. Pollefeys
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
TLDR
A novel SGM parameterization, which deploys different penalties depending on either positive or negative disparity changes in order to represent the object structures more discriminatively, is proposed.
Unsupervised Learning of Geometry From Videos With Edge-Aware Depth-Normal Consistency
TLDR
The proposed surface normal representation for unsupervised depth estimation framework is constrained to be compatible with predicted normals, yielding more robust geometry results and showing that the algorithm vastly outperforms state-of-the-art datasets, which demonstrates the benefits of the approach.
...
1
2
3
4
5
...