Deformable GANs for Pose-Based Human Image Generation

@article{Siarohin2018DeformableGF,
  title={Deformable GANs for Pose-Based Human Image Generation},
  author={Aliaksandr Siarohin and E. Sangineto and St{\'e}phane Lathuili{\`e}re and N. Sebe},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={3408-3416}
}
In this paper we address the problem of generating person images conditioned on a given pose. [...] Key Method In order to deal with pixel-to-pixel misalignments caused by the pose differences, we introduce deformable skip connections in the generator of our Generative Adversarial Network. Moreover, a nearest-neighbour loss is proposed instead of the common L1 and L2 losses in order to match the details of the generated image with the target image. We test our approach using photos of persons in different poses…Expand
Pose Guided Person Image Generation Based on Pose Skeleton Sequence and 3D Convolution
TLDR
This paper proposes a novel approach for pose transfer, which aims at transferring the pose of a given person to a target pose, and generates a sequence of different intermediate pose skeleton to capture the geometric constraints among the condition pose and target pose. Expand
Attention-guided GANs for human pose transfer
TLDR
This paper presents a novel generative adversarial network for the task of human pose transfer, which aims at transferring the pose of a given person to a target pose, and introduces an attention mechanism and proposes Pose-Guided Attention Blocks. Expand
Generation of Human Images with Clothing using Advanced Conditional Generative Adversarial Networks
TLDR
This paper investigated how Conditional GANs, along with Spectral Normalization (SN), could synthesize the new image of the target person given the image ofThe person and the target (novel) pose desired. Expand
Unpaired Pose Guided Human Image Generation
TLDR
An end-to-end trainable network under the generative adversarial framework is proposed that provides detailed control over the final appearance while not requiring paired training data and hence allows us to forgo the challenging problem of fitting 3D poses to 2D images. Expand
XingGAN for Person Image Generation
TLDR
A novel Generative Adversarial Network (XingGAN or CrossingGAN) for person image generation tasks, i.e., translating the pose of a given person to a desired one to mutually improve each other is proposed. Expand
Dense Pose Transfer
TLDR
This work proposes a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i.e. synthesize a new image of a person based on a single image of that person and theimage of a pose donor. Expand
A Robust Pose Transformational GAN for Pose Guided Person Image Synthesis
TLDR
This work proposes a simple yet effective pose transformation GAN by utilizing the Residual Learning method without any additional feature learning to generate a given human image in any arbitrary pose using effective data augmentation techniques and cleverly tuning the model. Expand
Loss Functions for Person Image Generation
TLDR
This paper extends the structural similarity (SSIM) index into a novel part-based SSIM loss to explicitly account for the articulated body structure and indicates that using different loss functions significantly impacts the generated person images and the proposed part- based SSIM lost is complementary to the prior losses and helps improve the performance. Expand
Spectral Normalization and Relativistic Adversarial Training for Conditional Pose Generation with Self-Attention
TLDR
This work base their approach on generative adversarial networks and leverage deformable skip connections to deal with pixel-to-pixel misalignments, self-attention to leverage complementary features in separate portions of the image, e.g., arms or legs, and spectral normalization to improve the quality of the synthesized images. Expand
Progressive and Aligned Pose Attention Transfer for Person Image Generation
This paper proposes a new generative adversarial network for pose transfer, i.e., transferring the pose of a given person to a target pose. We design a progressive generator which comprises aExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 31 REFERENCES
Pose Guided Person Image Generation
TLDR
The novel Pose Guided Person Generation Network (PG$^2$) that allows to synthesize person images in arbitrary poses, based on an image of that person and a novel pose, is proposed. Expand
Multi-View Image Generation from a Single-View
TLDR
This paper proposes a novel image generation model termed VariGANs, which combines the merits of the variational inference and the Generative Adversarial Networks (GANs), and generates the target image in a coarse-to-fine manner instead of a single pass which suffers from severe artifacts. Expand
The Pose Knows: Video Forecasting by Generating Pose Futures
TLDR
This work exploits human pose detectors as a free source of supervision and breaks the video forecasting problem into two discrete steps, and uses the structured space of pose as an intermediate representation to sidestep the problems that GANs have in generating video pixels directly. Expand
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks. Expand
Deformable Convolutional Networks
TLDR
This work introduces two new modules to enhance the transformation modeling capability of CNNs, namely, deformable convolution and deformable RoI pooling, based on the idea of augmenting the spatial sampling locations in the modules with additional offsets and learning the offsets from the target tasks, without additional supervision. Expand
Unlabeled Samples Generated by GAN Improve the Person Re-identification Baseline in Vitro
TLDR
A simple semisupervised pipeline that only uses the original training set without collecting extra data, which effectively improves the discriminative ability of learned CNN embeddings and proposes the label smoothing regularization for outliers (LSRO). Expand
Improved Techniques for Training GANs
TLDR
This work focuses on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic, and presents ImageNet samples with unprecedented resolution and shows that the methods enable the model to learn recognizable features of ImageNet classes. Expand
A Generative Model of People in Clothing
TLDR
The first image-based generative model of people in clothing for the full body is presented, which sidestep the commonly used complex graphics rendering pipeline and the need for high-quality 3D scans of dressed people and is learned from a large image database. Expand
A Discriminatively Learned CNN Embedding for Person Reidentification
TLDR
This article proposes a Siamese network that simultaneously computes the identification loss and verification loss and learns a discriminative embedding and a similarity measurement at the same time, thus taking full usage of the re-ID annotations. Expand
Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields
We present an approach to efficiently detect the 2D pose of multiple people in an image. The approach uses a nonparametric representation, which we refer to as Part Affinity Fields (PAFs), to learnExpand
...
1
2
3
4
...