• Corpus ID: 238856848

NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild

  title={NeRS: Neural Reflectance Surfaces for Sparse-view 3D Reconstruction in the Wild},
  author={Jason Y. Zhang and Gengshan Yang and Shubham Tulsiani and Deva Ramanan},
Recent history has seen a tremendous growth of work exploring implicit representations of geometry and radiance, popularized through Neural Radiance Fields (NeRF). Such works are fundamentally based on a (implicit) volumetric representation of occupancy, allowing them to model diverse scene structure including translucent objects and atmospheric obscurants. But because the vast majority of real-world scenes are composed of well-defined surfaces, we introduce a surface analog of such implicit… 
NeROIC: Neural Rendering of Objects from Online Image Collections
This work presents a novel method to acquire object representations from online image collections, capturing high-quality geometry and material properties of arbitrary objects from photographs with varying cameras, illumination, and backgrounds, and introduces a robust normal estimation technique which eliminates the effect of geometric noise while retaining crucial details.
BANMo: Building Animatable 3D Neural Models from Many Casual Videos
The key insight is to merge three schools of thought; classic deformable shape models that make use of articulated bones and blend skinning, volumetric neural radiance fields (NeRFs) that are amenable to gradient-based optimization, and canonical embeddings that generate correspondences between pixels and an articulated model.
NeRF-SR: High-Quality Neural Radiance Fields using Super-Sampling
NeRFSR can further boost the performance of super-sampling by a refinement network that leverages the estimated depth at hand to hallucinate details from related patches on an HR reference image, and generates high-quality results for novel view synthesis at HR on both synthetic and real-world datasets.
VoLux-GAN: A Generative Model for 3D Face Synthesis with HDRI Relighting
VoLux-GAN, a generative framework to synthesize 3D-aware faces with convincing relighting, is proposed, a volumetric HDRI relighting method that can efficiently accumulate albedo, diffuse and specular lighting contributions along each 3D ray for any desired HDR environmental map.
Semantic-Aware Implicit Neural Audio-Driven Video Portrait Generation
  • Xian Liu, Yinghao Xu, Qianyi Wu, Hang Zhou, Wayne Wu, Bolei Zhou
  • Computer Science, Engineering
  • 2022
Semantic-aware Speaking Portrait NeRF (SSPNeRF), which creates delicate audio-driven portraits using one unified set of NeRF through two semantic-aware modules, and renders more realistic video portraits compared to previous methods.


NeRFactor: Neural Factorization of Shape and Reflectance Under an Unknown Illumination
Qualitative and quantitative experiments show that NeRFactor outperforms classic and deep learning-based state of the art across various tasks.
D-NeRF: Neural Radiance Fields for Dynamic Scenes
D-NeRF is introduced, a method that extends neural radiance fields to a dynamic domain, allowing to reconstruct and render novel images of objects under rigid and non-rigid motions from a single camera moving around the scene.
NeuS: Learning Neural Implicit Surfaces by Volume Rendering for Multi-view Reconstruction
Experiments show that NeuS outperforms the state-of-the-arts in high-quality surface reconstruction, especially for objects and scenes with complex structures and self-occlusion, even for highly complex objects.
UNISURF: Unifying Neural Implicit Surfaces and Radiance Fields for Multi-View Reconstruction
This work shows that implicit surface models and radiance fields can be formulated in a unified way, enabling both surface and volume rendering using the same model, and outperforms NeRF in terms of reconstruction quality while performing on par with IDR without requiring masks.
Putting NeRF on a Diet: Semantically Consistent Few-Shot View Synthesis
DietNeRF improves the perceptual quality of few-shot view synthesis when learned from scratch, can render novel views with as few as one observed image when pre-trained on a multi-view dataset, and produces plausible completions of completely unobserved regions.
BARF: Bundle-Adjusting Neural Radiance Fields
Bundle-Adjusting Neural Radiance Fields (BARF) is proposed for training NeRF from imperfect (or even unknown) camera poses — the joint problem of learning neural 3D representations and registering camera frames and experiments show that BARF can effectively optimize the neural scene representations and resolve large camera pose misalignment at the same time.
Occupancy Networks: Learning 3D Reconstruction in Function Space
This paper proposes Occupancy Networks, a new representation for learning-based 3D reconstruction methods that encodes a description of the 3D output at infinite resolution without excessive memory footprint, and validate that the representation can efficiently encode 3D structure and can be inferred from various kinds of input.
Scene Representation Networks: Continuous 3D-Structure-Aware Neural Scene Representations
The proposed Scene Representation Networks (SRNs), a continuous, 3D-structure-aware scene representation that encodes both geometry and appearance, are demonstrated by evaluating them for novel view synthesis, few-shot reconstruction, joint shape and appearance interpolation, and unsupervised discovery of a non-rigid face model.
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
The 3D-R2N2 reconstruction framework outperforms the state-of-the-art methods for single view reconstruction, and enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).
DeepVoxels: Learning Persistent 3D Feature Embeddings
This work proposes DeepVoxels, a learned representation that encodes the view-dependent appearance of a 3D scene without having to explicitly model its geometry, based on a Cartesian 3D grid of persistent embedded features that learn to make use of the underlying3D scene structure.