Style Agnostic 3D Reconstruction via Adversarial Style Transfer

@article{Petersen2022StyleA3,
  title={Style Agnostic 3D Reconstruction via Adversarial Style Transfer},
  author={Felix Petersen and Bastian Goldluecke and Oliver Deussen and Hilde Kuehne},
  journal={2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},
  year={2022},
  pages={2273-2282}
}
Reconstructing the 3D geometry of an object from an image is a major challenge in computer vision. Recently introduced differentiable renderers can be leveraged to learn the 3D geometry of objects from 2D images, but those approaches require additional supervision to enable the renderer to produce an output that can be compared to the input image. This can be scene information or constraints such as object silhouettes, uniform backgrounds, material, texture, and lighting. In this paper, we… 
View on IEEE
arxiv.org
1 Citations

One Citation

Citation Type

Citation Type

GenDR: A Generalized Differentiable Renderer
TLDR
This work presents and studies a generalized family of differentiable renderers, which generalizes existing differentiability renderers like SoftRas and DIB-R, with an array of different smoothing distributions to cover a large spectrum of reasonable settings.

References

SHOWING 1-10 OF 36 REFERENCES
Learning to Generate and Reconstruct 3D Meshes with only 2D Supervision
TLDR
A unified framework tackling two problems: class-specific 3D reconstruction from a single image, and generation of new 3D shape samples, which is comparable or superior to state-of-the-art voxel-based approaches on quantitative metrics, while producing results that are visually more pleasing.
3D-RCNN: Instance-Level 3D Object Reconstruction via Render-and-Compare
TLDR
A differentiable Render-and-Compare loss is proposed that allows 3D shape and pose to be learned with 2D supervision and produces a compact 3D representation of the scene, which can be readily used for applications like autonomous driving.
DeformNet: Free-Form Deformation Network for 3D Shape Reconstruction from a Single Image
TLDR
The Free-Form Deformation layer is a powerful new building block for Deep Learning models that manipulate 3D data and DEFORMNET uses this FFD layer combined with shape retrieval for smooth and detail-preserving 3D reconstruction of qualitatively plausible point clouds with respect to a single query image.
Leveraging 2D Data to Learn Textured 3D Mesh Generation
TLDR
This work presents the first generative model of textured 3D meshes, and introduces a new generation process that guarantees no self-intersections arise, based on the physical intuition that faces should push one another out of the way as they move.
Learning to Predict 3D Objects with an Interpolation-based Differentiable Renderer
TLDR
A differentiable rendering framework which allows gradients to be analytically computed for all pixels in an image and to view foreground rasterization as a weighted interpolation of local properties and background rasterized as a distance-based aggregation of global geometry.
Learning Category-Specific Mesh Reconstruction from Image Collections
TLDR
A learning framework for recovering the 3D shape, camera, and texture of an object from a single image by incorporating texture inference as prediction of an image in a canonical appearance space and shows that semantic keypoints can be easily associated with the predicted shapes.
A Point Set Generation Network for 3D Object Reconstruction from a Single Image
TLDR
This paper addresses the problem of 3D reconstruction from a single image, generating a straight-forward form of output unorthordox, and designs architecture, loss function and learning paradigm that are novel and effective, capable of predicting multiple plausible 3D point clouds from an input image.
3D-R2N2: A Unified Approach for Single and Multi-view 3D Object Reconstruction
TLDR
The 3D-R2N2 reconstruction framework outperforms the state-of-the-art methods for single view reconstruction, and enables the 3D reconstruction of objects in situations when traditional SFM/SLAM methods fail (because of lack of texture and/or wide baseline).
Soft Rasterizer: A Differentiable Renderer for Image-Based 3D Reasoning
TLDR
This work proposes a truly differentiable rendering framework that is able to directly render colorized mesh using differentiable functions and back-propagate efficient supervision signals to mesh vertices and their attributes from various forms of image representations, including silhouette, shading and color images.
Perspective Transformer Nets: Learning Single-View 3D Object Reconstruction without 3D Supervision
TLDR
An encoder-decoder network with a novel projection loss defined by the projective transformation enables the unsupervised learning using 2D observation without explicit 3D supervision and shows superior performance and better generalization ability for 3D object reconstruction when the projection loss is involved.
...
1
2
3
4
...