GenDR: A Generalized Differentiable Renderer

  title={GenDR: A Generalized Differentiable Renderer},
  author={Felix Petersen and Bastian Goldluecke and Christian Borgelt and Oliver Deussen},
  journal={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
In this work, we present and study a generalized family of differentiable renderers. We discuss from scratch which components are necessary for differentiable rendering and formalize the requirements for each component. We instantiate our general differentiable renderer, which generalizes existing differentiable renderers like SoftRas and DIB-R, with an array of different smoothing distributions to cover a large spectrum of reasonable settings. We evaluate an array of differentiable renderer… 

Differentiable Collision Detection: a Randomized Smoothing Approach

This work introduces a generic andcient approach to compute the derivatives of collision detection for any pair of convex shapes, by notably leveraging randomized smoothing techniques which have shown to be particularly adapted to capture the derivative of non-smooth problems.



OpenDR: An Approximate Differentiable Renderer

This work describes a publicly available OpenDR framework that makes it easy to express a forward graphics model and then automatically obtain derivatives with respect to the model parameters and to optimize over them and demonstrates the power and simplicity of programming with OpenDR by using it to solve the problem of estimating human body shape from Kinect depth and RGB data.

Differentiable Rendering with Perturbed Optimizers

This work highlights the link between some well-known differentiable renderer formulations and randomly smoothed optimizers, and introduces differentiable perturbed renderers, and proposes a variance reduction mechanism to alleviate the computational burden inherent to perturbed optimizers and introduces an adaptive scheme to automatically adjust the smoothing parameters of the rendering process.

Differentiable Rendering: A Survey

This paper reviews existing literature and discusses the current state of differentiable rendering, its applications and open research problems.

Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning

This work proposes a truly differentiable rendering framework that is able to directly render colorized mesh using differentiable functions and back-propagate efficient supervision signals to mesh vertices and their attributes from various forms of image representations, including silhouette, shading and color images.

Reparameterizing discontinuous integrands for differentiable rendering

This work proposes a new technique for differentiating path-traced images with respect to scene parameters that affect visibility, including the position of cameras, light sources, and vertices in triangle meshes, and uses it to reconstruct the 3D geometry and materials of several real-world objects from a set of reference photographs.

Path-space differentiable rendering

It is shown how path integrals can be differentiated with respect to arbitrary differentiable changes of a scene and the design of new Monte Carlo estimators that offer significantly better efficiency than state-of-the-art methods in handling complex geometric discontinuities and light transport phenomena such as caustics are considered.

Differentiable surface splatting for point-based geometry processing

Applications of DSS to inverse rendering for geometry synthesis and denoising, where large scale topological changes, as well as small scale detail modifications, are accurately and robustly handled without requiring explicit connectivity, outperforming state-of-the-art techniques are demonstrated.

Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer

A differentiable rendering framework which allows gradients to be analytically computed for all pixels in an image and to view foreground rasterization as a weighted interpolation of local properties and background rasterized as a distance-based aggregation of global geometry.

Material Editing Using a Physically Based Rendering Network

This work proposes an end-to-end network architecture that replicates the forward image formation process to accomplish material editing, and demonstrates a rich set of visually plausible material editing examples and provides an extensive comparative study.

SDFDiff: Differentiable Rendering of Signed Distance Fields for 3D Shape Optimization

It is demonstrated that the proposed SDFDiff, a novel approach for image-based shape optimization using differentiable rendering of 3D shapes represented by signed distance functions, can be integrated with deep learning models, which opens up options for learning approaches on 3D objects without 3D supervision.