• Corpus ID: 238856848

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

@inproceedings{Zhang2021NeRSNR,
  title={NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild},
  author={Jason Y. Zhang and Gengshan Yang and Shubham Tulsiani and Deva Ramanan},
  booktitle={NeurIPS},
  year={2021}
}
Recent history has seen a tremendous growth of work exploring implicit representations of geometry and radiance, popularized through Neural Radiance Fields (NeRF). Such works are fundamentally based on a (implicit) volumetric representation of occupancy, allowing them to model diverse scene structure including translucent objects and atmospheric obscurants. But because the vast majority of real-world scenes are composed of well-defined surfaces, we introduce a surface analog of such implicit… 
View PDF on arXiv
View With Semantic Reader
11 Citations
Background Citations
8
Methods Citations
2

11 Citations

Citation Type

Citation Type

NeROIC: Neural Rendering of Objects from Online Image Collections
TLDR
This work presents a novel method to acquire object representations from online image collections, capturing high-quality geometry and material properties of arbitrary objects from photographs with varying cameras, illumination, and backgrounds, and introduces a robust normal estimation technique which eliminates the effect of geometric noise while retaining crucial details.
Towards Learning Neural Representations from Shadows
TLDR
This work proposes a graphics-inspired differentiable approach to render accurate shadows with volumetric rendering, predicting a shadow map that can be compared to the ground truth shadow and shows that neural rendering can localize the object and estimate coarse geometry.
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
TLDR
The key insight is to merge three schools of thought; classic deformable shape models that make use of articulated bones and blend skinning, volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and canonical embeddings that generate correspondences between pixels and an articulated model.
NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling
TLDR
NeRFSR can further boost the performance of super-sampling by a refinement network that leverages the estimated depth at hand to hallucinate details from related patches on an HR reference image, and generates high-quality results for novel view synthesis at HR on both synthetic and real-world datasets.
Topologically-Aware Deformation Fields for Single-View 3D Reconstruction
TLDR
This work presents a new framework for learning 3D object shapes and dense cross-object 3D correspondences from just an unaligned category-specific image collection, dubbed TARS, which achieves state-of-the-art reconstruction fidelity on several datasets.
Share With Thy Neighbors: Single-View Reconstruction by Cross-Instance Consistency
TLDR
This paper compares the approach, UNICORN, both on the diverse synthetic ShapeNet dataset - the classical benchmark for methods requiring multiple views as supervision - and on standard real-image benchmarks, for which most methods require known templates and silhouette annotations.
VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting
TLDR
VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting, is proposed, a volumetric HDRI relighting method that can efficiently accumulate albedo, diffuse and specular lighting contributions along each 3D ray for any desired HDR environmental map.
READ: Large-Scale Neural Scene Rendering for Autonomous Driving
TLDR
A large-scale neural rendering method is proposed to synthesized the autonomous driving scene (READ), which makes it possible to synthesize large- scale driving scenarios on a PC through a variety of sampling schemes.
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
TLDR
Semantic-aware Speaking Portrait NeRF (SSPNeRF), which creates delicate audio-driven portraits using one unified set of NeRF through two semantic-aware modules, and renders more realistic video portraits compared to previous methods.
FvOR: Robust Joint Shape and Pose Optimization for Few-view Object Reconstruction
Reconstructing an accurate 3D object model from a few image observations remains a challenging problem in computer vision. State-of-the-art approaches typically assume accurate camera poses as input,…
...
1
2
...

References

SHOWING 1-10 OF 68 REFERENCES
NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination
TLDR
Qualitative and quantitative experiments show that NeRFactor outperforms classic and deep learning-based state of the art across various tasks.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
TLDR
D-NeRF is introduced, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a single camera moving around the scene.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
TLDR
Experiments show that NeuS outperforms the state-of-the-arts in high-quality surface reconstruction, especially for objects and scenes with complex structures and self-occlusion, even for highly complex objects.
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
TLDR
This work shows that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering using the same model, and outperforms NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
BARF: Bundle-Adjusting Neural Radiance Fields
TLDR
Bundle-Adjusting Neural Radiance Fields (BARF) is proposed for training NeRF from imperfect (or even unknown) camera poses — the joint problem of learning neural 3D representations and registering camera frames and it is shown that coarse-to-fine registration is also applicable to NeRF.
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis
TLDR
DietNeRF improves the perceptual quality of few-shot view synthesis when learned from scratch, can render novel views with as few as one observed image when pre-trained on a multi-view dataset, and produces plausible completions of completely unobserved regions.
Occupancy Networks: Learning 3D Reconstruction in Function Space
TLDR
This paper proposes Occupancy Networks, a new representation for learning-based 3D reconstruction methods that encodes a description of the 3D output at infinite resolution without excessive memory footprint, and validate that the representation can efficiently encode 3D structure and can be inferred from various kinds of input.
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
TLDR
The proposed Scene Representation Networks (SRNs), a continuous, 3D-structure-aware scene representation that encodes both geometry and appearance, are demonstrated by evaluating them for novel view synthesis, few-shot reconstruction, joint shape and appearance interpolation, and unsupervised discovery of a non-rigid face model.
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
TLDR
The 3D-R2N2 reconstruction framework outperforms the state-of-the-art methods for single view reconstruction, and enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).
DeepVoxels: Learning Persistent 3D Feature Embeddings
TLDR
This work proposes DeepVoxels, a learned representation that encodes the view-dependent appearance of a 3D scene without having to explicitly model its geometry, based on a Cartesian 3D grid of persistent embedded features that learn to make use of the underlying3D scene structure.
...
1
2
3
4
5
...