Invertible Neural BRDF for Object Inverse Rendering

@inproceedings{Chen2020InvertibleNB,
  title={Invertible Neural BRDF for Object Inverse Rendering},
  author={Zhe Chen and S. Nobuhara and K. Nishino},
  booktitle={ECCV},
  year={2020}
}
We introduce a novel neural network-based BRDF model and a Bayesian framework for object inverse rendering, i.e., joint estimation of reflectance and natural illumination from a single image of an object of known geometry. The BRDF is expressed with an invertible neural network, namely, normalizing flow, which provides the expressive power of a high-dimensional representation, computational simplicity of a compact analytical model, and physical plausibility of a real-world BRDF. We extract the… Expand

Figures and Tables from this paper

Neural BRDF Representation and Importance Sampling
TLDR
This work proposes a compact neural network-based representation of BRDF data that combines high-accuracy reconstruction with efficient practical rendering via built-in interpolation of reflectance, and encode BRDFs as lightweight networks, and proposes a training scheme with adaptive angular sampling. Expand
Neural Ray-Tracing: Learning Surfaces and Reflectance for Relighting and View Synthesis
TLDR
The proposed approach for scene editing, relighting and reflectance estimation learned from synthetic and captured views on existing datasets is validated and outperforms existing neural rendering methods for relighting under known lighting conditions, and produces realistic reconstructions for relit and edited scenes. Expand
Q‐NET: A Network for Low‐dimensional Integrals of Neural Proxies
  • K. Subr
  • Computer Science
  • Comput. Graph. Forum
  • 2021
TLDR
A family of fixed networks that can calculate integrals of functions represented by sigmoidal universal approximators, called Q‐NETs, that operate on the parameters of the trained proxy and can calculate exact integrals over any subset of dimensions of the input domain. Expand
One Ring to Rule Them All: a simple solution to multi-view 3D-Reconstruction of shapes with unknown BRDF via a small Recurrent ResNet
TLDR
A special network design, a ResNet with a global feedback or ‘ring’ connection, which has a provable guarantee for finding a valid diffeomorphic shape parameterization is used, which enables a wide range of special-effect applications including novel-view-synthesis, relighting, material retouching, and shape exchange without additional coding effort. Expand

References

SHOWING 1-10 OF 49 REFERENCES
Deep inverse rendering for high-resolution SVBRDF estimation from an arbitrary number of images
TLDR
A key distinguishing feature of this framework is that it directly optimizes for the appearance parameters in a latent embedded space of spatially-varying appearance, such that no handcrafted heuristics are needed to regularize the optimization. Expand
Deep Reflectance Maps
TLDR
A convolutional neural architecture to estimate reflectance maps of specular materials in natural lighting conditions is proposed in an end-to-end learning formulation that directly predicts a reflectance map from the image itself. Expand
Reflectance and Natural Illumination from Single-Material Specular Objects Using Deep Learning
TLDR
A data-driven, learning-based approach trained on a very large dataset that estimates reflectance and illumination information from a single image depicting a single-material specular object from a given class under natural illumination is presented. Expand
Shape and Reflectance Estimation in the Wild
TLDR
This work directly tackle the problem of joint reflectance and geometry estimation under known but uncontrolled natural illumination by fully exploiting the surface orientation cues that become embedded in the appearance of the object by introducing two methods. Expand
A signal-processing framework for inverse rendering
TLDR
This work introduces a signal-processing framework which describes the reflected light field as a convolution of the lighting and BRDF, and expresses it mathematically as a product of spherical harmonic coefficients of the BRDF and the lighting. Expand
Inverse Path Tracing for Joint Material and Lighting Estimation
TLDR
A novel optimization method using a differentiable Monte Carlo renderer that computes derivatives with respect to the estimated unknown illumination and material properties enables joint optimization for physically correct light transport and material models using a tailored stochastic gradient descent. Expand
InverseRenderNet: Learning Single Image Inverse Rendering
  • Ye Yu, W. Smith
  • Computer Science
  • 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
TLDR
This work shows how to train a fully convolutional neural network to perform inverse rendering from a single, uncontrolled image, and believes this is the first attempt to use MVS supervision for learning inverse rendering. Expand
LIME: Live Intrinsic Material Estimation
TLDR
This work presents the first end-to-end approach for real-time material estimation for general object shapes with uniform material that only requires a single color image as input and proposes a novel highly efficient perceptual rendering loss that mimics real-world image formation and obtains intermediate results even during run time. Expand
Single image multimaterial estimation
TLDR
The idea is to strongly constrain the possible solutions so that the recovered reflectance conform with those of real-world materials and to demonstrate the effectiveness of the method on real and synthetic images. Expand
Efficient reflectance capture using an autoencoder
We propose a novel framework that automatically learns the lighting patterns for efficient reflectance acquisition, as well as how to faithfully reconstruct spatially varying anisotropic BRDFs andExpand
...
1
2
3
4
5
...