DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild

@article{Gler2017DenseRegFC,
  title={DenseReg: Fully Convolutional Dense Shape Regression In-the-Wild},
  author={Riza Alp G{\"u}ler and George Trigeorgis and Epameinondas Antonakos and Patrick Snape and Stefanos Zafeiriou and Iasonas Kokkinos},
  journal={2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2017},
  pages={2614-2623}
}
In this paper we propose to learn a mapping from image pixels into a dense template grid through a fully convolutional network. We formulate this task as a regression problem and train our network by leveraging upon manually annotated facial landmarks in-the-wild. We use such landmarks to establish a dense correspondence field between a three-dimensional object template and the input image, which then serves as the ground-truth for training our regression system. We show that we can combine… 

Figures and Tables from this paper

DensePose: Dense Human Pose Estimation in the Wild
TLDR
This work establishes dense correspondences between an RGB image and a surface-based representation of the human body, a task referred to as dense human pose estimation, and improves accuracy through cascading, obtaining a system that delivers highly-accurate results at multiple frames per second on a single gpu.
Learning Dense Facial Correspondences in Unconstrained Images
TLDR
This work presents a minimalists but effective neural network that computes dense facial correspondences in highly unconstrained RGB images and demonstrates successful per-frame processing under extreme pose variations, occlusions, and lighting conditions.
Hand pose estimation through semi-supervised and weakly-supervised learning
"Look Ma, No Landmarks!" - Unsupervised, Model-Based Dense Face Alignment
TLDR
This paper shows how to train an image-to-image network to predict dense correspondence between a face image and a 3D morphable model using only the model for supervision and shows that both geometric parameters and photometric parameters can be inferred directly from the correspondence map using linear least squares and the novel inverse spherical harmonic lighting model.
Stacked Dense U-Nets with Dual Transformers for Robust Face Alignment
TLDR
This work designs a novel scale aggregation network topology structure and a channel aggregation building block to improve the model's capacity without sacrificing the computational complexity and model size and shows that accurate 3D face alignment can assist pose-invariant face recognition where it achieves a new state-of-the-art accuracy on CFP-FP.
DeepHuman: 3D Human Reconstruction From a Single Image
TLDR
DeepHuman, an image-guided volume-to-volume translation CNN for 3D human reconstruction from a single RGB image, leverages a dense semantic representation generated from SMPL model as an additional input to reduce the ambiguities associated with the reconstruction of invisible areas.
Dense Pose Transfer
TLDR
This work proposes a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i.e. synthesize a new image of a person based on a single image of that person and theimage of a pose donor.
Slim DensePose: Thrifty Learning From Sparse Annotations and Motion Cues
TLDR
It is demonstrated that if annotations are collected in video frames, their efficacy can be multiplied for free by using motion cues, and that motion cues help much more when they are extracted from videos.
Learning Dense Correspondence from Synthetic Environments
TLDR
2D-3D mapping network models on synthetic data is shown to be a viable alternative to using real data and has a high generalisa-tion potential towards real-world data.
MoFA: Model-Based Deep Convolutional Face Autoencoder for Unsupervised Monocular Reconstruction
TLDR
A novel model-based deep convolutional autoencoder that addresses the highly challenging problem of reconstructing a 3D human face from a single in-the-wild color image and can be trained end-to-end in an unsupervised manner, which renders training on very large real world data feasible.
...
...

References

SHOWING 1-10 OF 146 REFERENCES
DenseReg : Fully Convolutional Dense Shape Regression Inthe-Wild Rıza
TLDR
The proposed system, called DenseReg, allows to estimate dense image-to-template correspondences in a fully convolutional manner and can provide useful correspondence information as a stand-alone system, while when used as an initialization for Statistical Deformable Models the authors obtain landmark localization results that largely outperform the current state-of-the-art on the challenging 300W benchmark.
Learning Dense Correspondence via 3D-Guided Cycle Consistency
TLDR
It is demonstrated that the end-to-end trained ConvNet supervised by cycle-consistency outperforms state-of-the-art pairwise matching methods in correspondence-related tasks.
Dense Human Body Correspondences Using Convolutional Networks
TLDR
This work uses a deep convolutional neural network to train a feature descriptor on depth map pixels, but crucially, rather than training the network to solve the shape correspondence problem directly, it trains it to solve a body region classification problem, modified to increase the smoothness of the learned descriptors near region boundaries.
Hand Pose Estimation through Weakly-Supervised Learning of a Rich Intermediate Representation
TLDR
A method for hand pose estimation based on a deep regressor trained on two different kinds of input that decreases error on joints over direct regression of joints from depth data by 15.7%.
Learning shape correspondence with anisotropic convolutional neural networks
TLDR
An intrinsic convolutional neural network architecture based on anisotropic diffusion kernels is introduced, which is term Anisotropic Convolutional Neural Network (ACNN), and is used to effectively learn intrinsic dense correspondences between deformable shapes in very challenging settings.
Deep Learning for Semantic Part Segmentation with High-Level Guidance
TLDR
A state-of-the-art semantic segmentation system is adapted to this task, and it is shown that a combination of a fully-convolutional Deep CNN system coupled with Dense CRF labelling provides excellent results for a broad range of object categories.
DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs
TLDR
This work addresses the task of semantic image segmentation with Deep Learning and proposes atrous spatial pyramid pooling (ASPP), which is proposed to robustly segment objects at multiple scales, and improves the localization of object boundaries by combining methods from DCNNs and probabilistic graphical models.
Convolutional Pose Machines
TLDR
This work designs a sequential architecture composed of convolutional networks that directly operate on belief maps from previous stages, producing increasingly refined estimates for part locations, without the need for explicit graphical model-style inference in structured prediction tasks such as articulated pose estimation.
Fully Convolutional Networks for Semantic Segmentation
TLDR
It is shown that convolutional networks by themselves, trained end- to-end, pixels-to-pixels, improve on the previous best result in semantic segmentation.
Metric Regression Forests for Correspondence Estimation
TLDR
A new method for inferring dense data to model correspondences, focusing on the application of human pose estimation from depth images, that leads to correspondences that are considerably more accurate than state of the art, using far fewer training images.
...
...