Real-time Global Illumination Decomposition of Videos

  title={Real-time Global Illumination Decomposition of Videos},
  author={Abhimitra Meka and Mohammad Shafiei and Michael Zollh{\"o}fer and Christian Richardt and Christian Theobalt},
  journal={ACM Trans. Graph.},
Fig. 1. We propose the first approach for the real-time decomposition of a video into direct and indirect illumination components. Our approach decomposes a video (left) into its reflectance, direct illumination, and multiple indirect illumination layers (middle) that explain the light transport in the scene up to the first bounce. This enables various real-time appearance editing applications with interactive user feedback, such as inter-reflection consistent recoloring (right). 


Intrinsic Video
This work defines intrinsic video estimation as the problem of extracting temporally coherent albedo and shading from video alone and shows that it substantially outperforms existing single-frame intrinsic image methods. Expand
What Is a Theory?
It is shown that all theories in propositional calculus can be expressed in this framework and that cuts can always be eliminated with such theories. Expand
A Variational Model for Intrinsic Light Field Decomposition
A novel variational model for intrinsic light field decomposition is presented, which is performed on four-dimensional ray space instead of a traditional 2D image, and the advantages of this approach with respect to intrinsic image decomposition and specular removal are demonstrated. Expand
Optimization with Sparsity-Inducing Penalties
This monograph covers proximal methods, block-coordinate descent, reweighted l2-penalized techniques, working-set and homotopy methods, as well as non-convex formulations and extensions, and provides an extensive set of experiments to compare various algorithms from a computational point of view. Expand
Inverse Path Tracing for Joint Material and Lighting Estimation
A novel optimization method using a differentiable Monte Carlo renderer that computes derivatives with respect to the estimated unknown illumination and material properties enables joint optimization for physically correct light transport and material models using a tailored stochastic gradient descent. Expand
Deep Hybrid Real and Synthetic Training for Intrinsic Decomposition
This paper proposes a novel hybrid approach to train a deep convolutional neural network using a CNN on both synthetic and real images, and trains the network by enforcing it to produce the same reflectance for a pair of images of the same real-world scene with different illuminations. Expand
Efficient Palette-based
  • 2018
Efficient palette-based decomposition and recoloring of images via RGBXY-space geometry
An extremely scalable and efficient yet simple palette-based image decomposition algorithm that can decompose 6 MP images into layers in 20 milliseconds and a real-time layer decomposition tool in which users can interactively edit the palette to adjust the layers. Expand
Learning to reconstruct shape and spatially-varying reflectance from a single image
This work demonstrates that it can recover non-Lambertian, spatially-varying BRDFs and complex geometry belonging to any arbitrary shape class, from a single RGB image captured under a combination of unknown environment illumination and flash lighting. Expand
Practical SVBRDF
  • 2018