Occupancy Networks: Learning 3D Reconstruction in Function Space

@article{Mescheder2019OccupancyNL,
  title={Occupancy Networks: Learning 3D Reconstruction in Function Space},
  author={Lars M. Mescheder and Michael Oechsle and Michael Niemeyer and Sebastian Nowozin and Andreas Geiger},
  journal={2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2019},
  pages={4455-4465}
}
With the advent of deep neural networks, learning-based approaches for 3D reconstruction have gained popularity. [] Key Method Occupancy networks implicitly represent the 3D surface as the continuous decision boundary of a deep neural network classifier. In contrast to existing approaches, our representation encodes a description of the 3D output at infinite resolution without excessive memory footprint. We validate that our representation can efficiently encode 3D structure and can be inferred from various…

Figures and Tables from this paper

A Simple and Scalable Shape Representation for 3D Reconstruction
TLDR
This work shows that high quality 3D reconstruction is achievable using a linear decoder, obtained from principal component analysis on the signed distance function (SDF) of the surface, and that this approach is competitive with state-of-the-art methods.
Occupancy Flow: 4D Reconstruction by Learning Particle Dynamics
TLDR
This work presents Occupancy Flow, a novel spatio-temporal representation of time-varying 3D geometry with implicit correspondences which can be used for interpolation and reconstruction tasks, and believes that Occupancy flow is a promising new 4D representation which will be useful for a variety of spatio/temporal reconstruction tasks.
H3D-Net: Few-Shot High-Fidelity 3D Head Reconstruction
TLDR
This paper endows coordinate-based representations with a probabilistic shape prior that enables faster convergence and better generalization when using few input images, and achieves high-fidelity head reconstructions with a high level of detail that consistently outperforms both state-of-the-art 3D Morphable Models methods in the few-shot scenario, and nonparametric methods when large sets of views are available.
D-OccNet: Detailed 3D Reconstruction Using Cross-Domain Learning
TLDR
This work first converts the single view 2D image into a simpler point cloud representation, and then reconstruct a 3D surface from it, which outperforms Occupancy Networks in terms of visual quality and details captured in the 3D reconstruction.
MouldingNet: Deep-learning for 3D Object Reconstruction
TLDR
Bilateral convolutional layers are proposed to propose a novel end-to-end deep auto-encoder architecture to efficiently encode and reconstruct 3D point clouds and demonstrate its accuracy by evaluating across the shapenet and modelnet datasets.
Learning Compositional Shape Priors for Few-Shot 3D Reconstruction
TLDR
This work experimentally demonstrates that naive baselines fail in this few-shot learning setting, and proposes three ways to learn a class-specific global shape prior, directly from data, that capture multi-scale information about the 3D shape, and account for intra-class variability by virtue of an implicit compositional structure.
Learning to Infer Implicit Surfaces without 3D Supervision
TLDR
A novel ray-based field probing technique for efficient image-to-field supervision, as well as a general geometric regularizer for implicit surfaces, which provides natural shape priors in unconstrained regions are proposed.
Dynamic Plane Convolutional Occupancy Networks
TLDR
The proposed Dynamic Plane Convolutional Occupancy Networks, a novel implicit representation pushing further the quality of 3D surface reconstruction, shows superior performance in surface reconstruction from unoriented point clouds in ShapeNet as well as an indoor scene dataset.
Unsupervised Learning of 3D Object Categories from Videos in the Wild
TLDR
A new neural network design is proposed, called warp-conditioned ray embedding (WCR), which significantly improves reconstruction while obtaining a detailed implicit representation of the object surface and texture, also compensating for the noise in the initial SfM reconstruction that bootstrapped the learning process.
Higher-Order Function Networks for Learning Composable 3D Object Representations
TLDR
A new approach to 3D object representation where a neural network encodes the geometry of an object directly into the weights and biases of a second 'mapping' network, which can be used to reconstruct an object by applying its encoded transformation to points randomly sampled from a simple geometric space.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 86 REFERENCES
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
TLDR
The 3D-R2N2 reconstruction framework outperforms the state-of-the-art methods for single view reconstruction, and enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).
RayNet: Learning Volumetric 3D Reconstruction with Ray Potentials
TLDR
This paper proposes RayNet, which combines a CNN that learns view-invariant feature representations with an MRF that explicitly encodes the physics of perspective projection and occlusion and trains RayNet end-to-end using empirical risk minimization.
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
TLDR
This paper addresses the problem of 3D reconstruction from a single image, generating a straight-forward form of output unorthordox, and designs architecture, loss function and learning paradigm that are novel and effective, capable of predicting multiple plausible 3D point clouds from an input image.
OctNetFusion: Learning Depth Fusion from Data
TLDR
This paper presents a novel 3D CNN architecture that learns to predict an implicit surface representation from the input depth maps and significantly outperforms the traditional volumetric fusion approach in terms of noise reduction and outlier suppression.
Learning Shape Priors for Single-View 3D Completion and Reconstruction
TLDR
The proposed ShapeHD pushes the limit of single-view shape completion and reconstruction by integrating deep generative models with adversarially learned shape priors, penalizing the model only if its output is unrealistic, not if it deviates from the ground truth.
Deep Marching Cubes: Learning Explicit Surface Representations
TLDR
This paper demonstrates that the marching cubes algorithm is not differentiable and proposes an alternative differentiable formulation which is inserted as a final layer into a 3D convolutional neural network, and proposes a set of loss functions which allow for training the model with sparse point supervision.
Shape Completion Using 3D-Encoder-Predictor CNNs and Shape Synthesis
TLDR
A data-driven approach to complete partial 3D shapes through a combination of volumetric deep neural networks and 3D shape synthesis and a 3D-Encoder-Predictor Network (3D-EPN) which is composed of 3D convolutional layers.
Learning Category-Specific Mesh Reconstruction from Image Collections
TLDR
A learning framework for recovering the 3D shape, camera, and texture of an object from a single image by incorporating texture inference as prediction of an image in a canonical appearance space and shows that semantic keypoints can be easily associated with the predicted shapes.
Deep Level Sets: Implicit Surface Representations for 3D Shape Inference
TLDR
An end-to-end trainable model is proposed that directly predicts implicit surface representations of arbitrary topology by optimising a novel geometric loss function and incorporating this in a deep end- to-end learning framework by introducing a variational shape inference formulation.
DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation
TLDR
This work introduces DeepSDF, a learned continuous Signed Distance Function (SDF) representation of a class of shapes that enables high quality shape representation, interpolation and completion from partial and noisy 3D input data.
...
1
2
3
4
5
...