Enabling Viewpoint Learning through Dynamic Label Generation

@article{Schelling2021EnablingVL,
  title={Enabling Viewpoint Learning through Dynamic Label Generation},
  author={Michael Schelling and Pedro Hermosilla and Pere-Pau V{\'a}zquez and Timo Ropinski},
  journal={Computer Graphics Forum},
  year={2021},
  volume={40}
}
Optimal viewpoint prediction is an essential task in many computer graphics applications. Unfortunately, common viewpoint qualities suffer from two major drawbacks: dependency on clean surface meshes, which are not always available, and the lack of closed‐form expressions, which requires a costly search involving rendering. To overcome these limitations we propose to separate viewpoint selection from rendering through an end‐to‐end learning approach, whereby we reduce the influence of the mesh… 
1 Citations
Learning Human Viewpoint Preferences from Sparsely Annotated Models
TLDR
This work introduces a large-scale crowdsourced data set, which contains 58 k annotated viewpoints for 3220 ModelNet40 models, and derives a neural view quality measure abiding to human preferences, which is the first deep learning approach to predict a viewquality measure solely based on human preferences.

References

SHOWING 1-10 OF 81 REFERENCES
Render for CNN: Viewpoint Estimation in Images Using CNNs Trained with Rendered 3D Model Views
TLDR
A scalable and overfit-resistant image synthesis pipeline, together with a novel CNN specifically tailored for the viewpoint estimation task, is proposed that can significantly outperform state-of-the-art methods on PASCAL 3D+ benchmark.
CNNs Based Viewpoint Estimation for Volume Visualization
TLDR
An overfit-resistant image rendering pipeline is designed to generate the training images with accurate viewpoint annotations, and a category-specific viewpoint classification network is trained to estimate the viewpoint for the given rendered image.
Deep learning-based viewpoint recommendation in volume visualization
TLDR
This work proposes an implicit model for the best viewpoint recommendation of volume visualization with CNN-based models to learn the traditional scoring method and user preference and flexibly realize a user preference-based best viewpoint selection in volume visualization.
Spherical Regression: Learning Viewpoints, Surface Normals and 3D Rotations on N-Spheres
TLDR
By introducing a spherical exponential mapping on n-spheres at the regression output, this work obtains well-behaved gradients, leading to stable training and shows how the spherical regression can be utilized for several computer vision challenges, specifically viewpoint estimation, surface normal estimation and 3D rotation estimation.
RenderNet: A deep convolutional network for differentiable rendering from 3D shapes
TLDR
RenderNet is presented, a differentiable rendering convolutional network with a novel projection unit that can render 2D images from 3D shapes with high performance and can be used in inverse rendering tasks to estimate shape, pose, lighting and texture from a single image.
Viewpoint Selection for Photographing Architectures
TLDR
Combining 2D image features extracted from images with 3D geometric features computed on the 3D models can result in more reliable evaluation of viewpoint quality, and is shown to achieve superior performance over using solely 2D or 3D features.
3D ShapeNets: A deep representation for volumetric shapes
TLDR
This work proposes to represent a geometric 3D shape as a probability distribution of binary variables on a 3D voxel grid, using a Convolutional Deep Belief Network, and shows that this 3D deep representation enables significant performance improvement over the-state-of-the-arts in a variety of tasks.
Category‐Specific Salient View Selection via Deep Convolutional Neural Networks
TLDR
A new framework to determine up front orientations and detect salient views of 3D models and effectively combines category‐specific scores and classical evaluations to produce a data‐driven viewpoint saliency map is presented.
Learning in an Uncertain World: Representing Ambiguity Through Multiple Hypotheses
TLDR
This work proposes a frame-work for reformulating existing single-prediction models as multiple hypothesis prediction (MHP) models and an associated meta loss and optimization procedure to train them, and finds that MHP models outperform their single-hypothesis counterparts in all cases and expose valuable insights into the variability of predictions.
...
...