Unsupervised Creation of Parameterized Avatars

@article{Wolf2017UnsupervisedCO,
  title={Unsupervised Creation of Parameterized Avatars},
  author={Lior Wolf and Yaniv Taigman and Adam Polyak},
  journal={2017 IEEE International Conference on Computer Vision (ICCV)},
  year={2017},
  pages={1539-1547}
}
We study the problem of mapping an input image to a tied pair consisting of a vector of parameters and an image that is created using a graphical engine from the vector of parameters. The mapping's objective is to have the output image as similar as possible to the input image. During training, no supervision is given in the form of matching inputs and outputs. This learning problem extends two literature problems: unsupervised domain adaptation and cross domain transfer. We define a… 

Figures and Tables from this paper

Unsupervised Generation of Free-Form and Parameterized Avatars
TLDR
Two unsupervised learning problems involving the task of mapping images between different domains are studied, generalization bounds that are based on discrepancy are defined, and a GAN is employed to implement network solutions that correspond to these bounds.
Unsupervised generation of artistic representations
TLDR
A novel approach to the unsupervised domain adaptation problem is presented, allowing us to successfully generate avatars from photos, and the proposed solution is highly flexible, which allows tuning of individual elements of the system to achieve different visual results.
XGAN: Unsupervised Image-to-Image Translation for many-to-many Mappings
TLDR
XGAN ("Cross-GAN"), a dual adversarial autoencoder, is introduced, which captures a shared representation of the common domain semantic content in an unsupervised way, while jointly learning the domain-to-domain image translations in both directions.
Improving Shape Deformation in Unsupervised Image-to-Image Translation
TLDR
This work introduces a discriminator with dilated convolutions that is able to use information from across the entire image to train a more context-aware generator, coupled with a multi-scale perceptual loss that is better able to represent error in the underlying shape of objects.
NAM - UNSUPERVISED CROSS-DOMAIN IMAGE MAP-
TLDR
NAM relies on a pretrained generative model of the source domain, and aligns each target image with an image sampled from the source distribution while jointly optimizing the domain mapping function.
Multimodal Unsupervised Image-to-Image Translation
TLDR
A Multimodal Unsupervised Image-to-image Translation (MUNIT) framework that assumes that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties.
Estimating the Success of Unsupervised Image to Image Translation
TLDR
This work proposes a novel bound for predicting the success of unsupervised cross domain mapping methods, which is motivated by the recently proposed Simplicity Principle and can be applied both in expectation, for comparing hyperparameters and for selecting a stopping criterion per sample in order to predict thesuccess of a specific cross-domain translation.
Fader Networks: Manipulating Images by Sliding Attributes
TLDR
A new encoder-decoder architecture that is trained to reconstruct images by disentangling the salient information of the image and the values of attributes directly in the latent space is introduced, which results in much simpler training schemes and nicely scales to multiple attributes.
Face-to-Parameter Translation for Game Character Auto-Creation
TLDR
Experimental results demonstrate that the proposed method achieves a high degree of generation similarity between the input face photo and the created in-game character in terms of both global appearance and local details.
Adversarial Information Factorization
TLDR
A novel generative model architecture designed to learn representations for images that factor out a single attribute from the rest of the representation is proposed, which achieves competitive scores, with state of the art, on a facial attribute classification task.
...
...

References

SHOWING 1-10 OF 33 REFERENCES
Unsupervised Cross-Domain Image Generation
TLDR
The Domain Transfer Network (DTN) is presented, which employs a compound loss function that includes a multiclass GAN loss, an f-constancy component, and a regularizing component that encourages G to map samples from T to themselves.
Learning to generate chairs with convolutional neural networks
TLDR
This work trains a generative convolutional neural network which is able to generate images of objects given object type, viewpoint, and color and shows that the network can be used to find correspondences between different chairs from the dataset, outperforming existing approaches on this task.
Image Style Transfer Using Convolutional Neural Networks
TLDR
A Neural Algorithm of Artistic Style is introduced that can separate and recombine the image content and style of natural images and provide new insights into the deep image representations learned by Convolutional Neural Networks and demonstrate their potential for high level image synthesis and manipulation.
Deep Convolutional Inverse Graphics Network
This paper presents the Deep Convolution Inverse Graphics Network (DC-IGN), a model that aims to learn an interpretable representation of images, disentangled with respect to three-dimensional scene
DRAW: A Recurrent Neural Network For Image Generation
TLDR
The Deep Recurrent Attentive Writer neural network architecture for image generation substantially improves on the state of the art for generative models on MNIST, and, when trained on the Street View House Numbers dataset, it generates images that cannot be distinguished from real data with the naked eye.
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
TLDR
This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.
Unsupervised Learning of 3D Structure from Images
TLDR
This paper learns strong deep generative models of 3D structures, and recovers these structures from 3D and 2D images via probabilistic inference, demonstrating for the first time the feasibility of learning to infer 3D representations of the world in a purely unsupervised manner.
Generating Images with Perceptual Similarity Metrics based on Deep Networks
TLDR
A class of loss functions, which are called deep perceptual similarity metrics (DeePSiM), are proposed that compute distances between image features extracted by deep neural networks and better reflects perceptually similarity of images and thus leads to better results.
A Theory of Output-Side Unsupervised Domain Adaptation
TLDR
This work considers the complementary problem in which the unlabeled samples are given post mapping, i.e., the authors are given the outputs of the mapping of unknown samples from the shifted domain.
...
...