Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis

@article{Liu2019LiquidWG,
  title={Liquid Warping GAN: A Unified Framework for Human Motion Imitation, Appearance Transfer and Novel View Synthesis},
  author={Wen Liu and Zhixin Piao and Jie Min and Wenhan Luo and Lin Ma and Shenghua Gao},
  journal={2019 IEEE/CVF International Conference on Computer Vision (ICCV)},
  year={2019},
  pages={5903-5912}
}
  • Wen Liu, Zhixin Piao, +3 authors Shenghua Gao
  • Published 26 September 2019
  • Computer Science, Engineering
  • 2019 IEEE/CVF International Conference on Computer Vision (ICCV)
We tackle the human motion imitation, appearance transfer, and novel view synthesis within a unified framework, which means that the model once being trained can be used to handle all these tasks. The existing task-specific methods mainly use 2D keypoints (pose) to estimate the human body structure. However, they only expresses the position information with no abilities to characterize the personalized shape of the individual person and model the limbs rotations. In this paper, we propose to… Expand
Liquid Warping GAN with Attention: A Unified Framework for Human Image Synthesis
TLDR
This work proposes to use a 3D body mesh recovery module to disentangle the pose and shape and builds a new dataset, namely iPER, for the evaluation of these three tasks, which tackle human image synthesis within a unified framework. Expand
RIN: Textured Human Model Recovery and Imitation with a Single Image
TLDR
This paper proposes RIN, a novel volume-based framework for reconstructing a textured 3D model from a single picture and imitating a subject with the generated model, using a U-Net-like front-to-back translation network. Expand
Reposing Humans by Warping 3D Features
TLDR
The problem of reposing an image of a human into any desired novel pose is addressed and a dense feature volume is implicitly learned from human images, which lends itself to simple and intuitive manipulation through explicit geometric warping. Expand
Creating and Reenacting Controllable 3D Humans with Differentiable Rendering
TLDR
This paper proposes a new end-to-end neural rendering architecture to transfer appearance and reenact human actors by taking advantages of both differentiable rendering and the 3D parametric model, which allows controlling the human synthesis from both pose and rendering parameters. Expand
Focus and retain: Complement the Broken Pose in Human Image Synthesis
  • Pu Ge, Qiushi Huang, +4 authors Zhun Sun
  • Computer Science
  • 2021 IEEE Winter Conference on Applications of Computer Vision (WACV)
  • 2021
TLDR
A feature-level flow attention module and an Enhancer Network is proposed to tackle the problem of how to generate an image of a specific style with that target pose, which outperforms some state-of-the-arts including Guide-Pixe2PixE, Global-Flow-Local-Attn, and CocosNet. Expand
Human Motion Transfer with 3D Constraints and Detail Enhancement
We propose a new method for realistic human motion transfer using a generative adversarial network (GAN), which generates a motion video of a target character imitating actions of a source character,Expand
Neural Actor: Neural Free-view Synthesis of Human Actors with Pose Control
TLDR
Experiments demonstrate that the proposed Neural Actor method achieves better quality than the state-of-the-arts on playback as well as novel pose synthesis, and can even generalize well to new poses that starkly differ from the training poses. Expand
Pose-Guided Human Animation from a Single Image in the Wild
TLDR
A compositional neural network is designed that predicts the silhouette, garment labels, and textures of a person and is used to synthesize human animations that can preserve the identity and appearance of the person in a temporally coherent way without any fine-tuning of the network on the testing scene. Expand
Learning Realistic Human Reposing using Cyclic Self-Supervision with 3D Shape, Pose, and Appearance Consistency
TLDR
A self-super supervised framework named SPICE (Self-supervised Person Image CrEation) that closes the image quality gap with supervised methods and achieves state-of-the-art performance on the DeepFashion dataset. Expand
A Shape-Aware Retargeting Approach to Transfer Human Motion and Appearance in Monocular Videos
TLDR
This work proposes a shape-aware approach based on a hybrid image-based rendering technique that exhibits competitive visual retargeting quality compared to state-of-the-art neural rendering approaches and presents a new video retargeted benchmark dataset to evaluate the task of synthesizing people’s videos. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 54 REFERENCES
Human Appearance Transfer
TLDR
The proposed architecture can be profiled to automatically generate images of a person dressed with different clothing transferred from a person in another image, opening paths for applications in entertainment and photo-editing, or affordable online shopping of clothing. Expand
View Extrapolation of Human Body from a Single Image
TLDR
A novel deep learning based pipeline that explicitly estimates and leverages the geometry of the underlying human body is proposed that is able to factor out the space of data variation and makes learning at each step much easier. Expand
Soft-Gated Warping-GAN for Pose-Guided Person Image Synthesis
TLDR
Human perceptual studies and quantitative evaluations demonstrate the superiority of the Warping-GAN that significantly outperforms all existing methods on two large datasets and is light-weight and flexible enough to be injected into any networks. Expand
Dense Pose Transfer
TLDR
This work proposes a combination of surface-based pose estimation and deep generative models that allows us to perform accurate pose transfer, i.e. synthesize a new image of a person based on a single image of that person and theimage of a pose donor. Expand
Multistage Adversarial Losses for Pose-Based Human Image Synthesis
  • Chenyang Si, W. Wang, Liang Wang, T. Tan
  • Computer Science
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
TLDR
This paper proposes a pose-based human image synthesis method which can keep the human posture unchanged in novel viewpoints and adopt multistage adversarial losses separately for the foreground and background generation, which fully exploits the multi-modal characteristics of generative loss to generate more realistic looking images. Expand
GANimation: Anatomically-aware Facial Animation from a Single Image
TLDR
A novel GAN conditioning scheme based on Action Units (AU) annotations, which describes in a continuous manifold the anatomical facial movements defining a human expression, and proposes a fully unsupervised strategy to train the model, that only requires images annotated with their activated AUs. Expand
Keep It SMPL: Automatic Estimation of 3D Human Pose and Shape from a Single Image
TLDR
The first method to automatically estimate the 3D pose of the human body as well as its 3D shape from a single unconstrained image is described, showing superior pose accuracy with respect to the state of the art. Expand
End-to-End Recovery of Human Shape and Pose
TLDR
This work introduces an adversary trained to tell whether human body shape and pose parameters are real or not using a large database of 3D human meshes, and produces a richer and more useful mesh representation that is parameterized by shape and 3D joint angles. Expand
SwapNet: Image Based Garment Transfer
TLDR
This work presents Swapnet, a framework to transfer garments across images of people with arbitrary body pose, shape, and clothing, and proposes a novel weakly-supervised approach that generates training pairs from a single image via data augmentation. Expand
Synthesizing Images of Humans in Unseen Poses
TLDR
A modular generative neural network is presented that synthesizes unseen poses using training pairs of images and poses taken from human action videos, separates a scene into different body part and background layers, moves body parts to new locations and refines their appearances, and composites the new foreground with a hole-filled background. Expand
...
1
2
3
4
5
...