• Corpus ID: 202767986

First Order Motion Model for Image Animation

@article{Siarohin2019FirstOM,
  title={First Order Motion Model for Image Animation},
  author={Aliaksandr Siarohin and St{\'e}phane Lathuili{\`e}re and S. Tulyakov and Elisa Ricci and N. Sebe},
  journal={ArXiv},
  year={2019},
  volume={abs/2003.00196}
}
Image animation consists of generating a video sequence so that an object in a source image is animated according to the motion of a driving video. Our framework addresses this problem without using any annotation or prior information about the specific object to animate. Once trained on a set of videos depicting objects of the same category (e.g. faces, human bodies), our method can be applied to any object of this class. To achieve this, we decouple appearance and motion information using a… 
Image Animation with Keypoint Mask
TLDR
This work extracts the structure from a keypoint heatmap, without an explicit motion representation, and uses it to warp the image according to the video, by a deep generator.
Image Animation with Perturbed Masks
TLDR
This work presents a novel approach for image-animation of a source image by a driving video, both depicting the same type of object, and is shown to greatly outperform the state of the art methods on multiple benchmarks.
Motion Representations for Articulated Animation
TLDR
This work proposes novel motion representations for animating articulated objects consisting of distinct parts in a completely unsupervised manner, and can animate a variety of objects, surpassing previous methods by a large margin on existing benchmarks.
Animating Pictures with Eulerian Motion Fields
TLDR
This paper proposes a novel video looping technique that flows features both for-ward and backward in time and then blends the results, demonstrating the effectiveness and robustness of this fully automatic method for converting a still image into a realistic animated looping video.
Image Animations on Driving Videos with DeepFakes and Detecting DeepFakes Generated Animations
The concept of image animation is to create a video or animation such that an object from an image is animated as per the motion of driving video. We plan to analyze with minor modifications of an
Understanding Object Dynamics for Interactive Image-to-Video Synthesis
TLDR
This generative model learns to infer natural object dynamics as a response to user interaction and learns about the interrelations between different object body regions and can transfer dynamics onto novel unseen object instances.
Self-appearance-aided Differential Evolution for Motion Transfer
  • Peirong Liu, Rui Wang, +5 authors Ser-Nam Lim
  • Computer Science
    ArXiv
  • 2021
TLDR
Model generality analysis indicates that the proposed framework for unsupervised motion transfer outperforms the state-ofthe-arts by a significant margin, across six benchmarks varying from human faces, human bodies to robots and cartoon characters.
AniVid: A Novel Anime Video Dataset with Applications in Animation
Automating steps of the animation production process using AI-based tools would ease the workload of Japanese animators. Although there have been recent advances in the automatic animation of still
Pose-Guided Human Animation from a Single Image in the Wild
TLDR
A compositional neural network is designed that predicts the silhouette, garment labels, and textures of a person and is used to synthesize human animations that can preserve the identity and appearance of the person in a temporally coherent way without any fine-tuning of the network on the testing scene.
Sparse to Dense Motion Transfer for Face Image Animation
  • Ruiqi Zhao, Tianyi Wu, Guodong Guo
  • Computer Science
    2021 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW)
  • 2021
TLDR
This work develops an efficient and effective method for motion transfer from sparse landmarks to the face image, and combines global and local motion estimation in a unified model to faithfully transfer the motion.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 46 REFERENCES
Animating Arbitrary Objects via Deep Motion Transfer
TLDR
This paper introduces a novel deep learning framework for image animation that generates a video in which the target object is animated according to the driving sequence through a deep architecture that decouples appearance and motion information.
Learning to Forecast and Refine Residual Motion for Image-to-Video Generation
TLDR
This work trains networks to learn residual motion between the current and future frames, which avoids learning motion-irrelevant details and proposes a two-stage generation framework where videos are generated from structures and then refined by temporal signals.
Everybody Dance Now
This paper presents a simple method for “do as I do” motion transfer: given a source video of a person dancing, we can transfer that performance to a novel (amateur) target after only a few minutes
DwNet: Dense warp-based network for pose-guided human video generation
TLDR
This paper focuses on human motion transfer - generation of a video depicting a particular subject, observed in a single image, performing a series of motions exemplified by an auxiliary (driving) video.
Textured Neural Avatars
TLDR
A system for learning full body neural avatars, i.e. deep networks that produce full body renderings of a person for varying body pose and varying camera pose, that is capable of learning to generate realistic renderings while being trained on videos annotated with 3D poses and foreground masks is presented.
Generating Videos with Scene Dynamics
TLDR
A generative adversarial network for video with a spatio-temporal convolutional architecture that untangles the scene's foreground from the background is proposed that can generate tiny videos up to a second at full frame rate better than simple baselines.
Transformation-Based Models of Video Sequences
TLDR
This work proposes a simple unsupervised approach for next frame prediction in video that compares favourably against more sophisticated ones on the UCF-101 data set, while also being more efficient in terms of the number of parameters and computational cost.
Coordinate-based Texture Inpainting for Pose-Guided Image Generation
TLDR
A new deep learning approach to pose-guided resynthesis of human photographs using a fully-convolutional architecture with deformable skip connections guided by the estimated correspondence field and a new inpainting method that completes the texture of the human body.
Displaced dynamic expression regression for real-time facial tracking and animation
TLDR
This work presents a fully automatic approach to real-time facial tracking and animation with a single video camera that learns a generic regressor from public image datasets to infer accurate 2D facial landmarks as well as the 3D facial shape from 2D video frames.
Unsupervised Learning of Object Landmarks through Conditional Image Generation
TLDR
This work proposes a method for learning landmark detectors for visual objects (such as the eyes and the nose in a face) without any manual supervision and introduces a tight bottleneck in the geometry-extraction process that selects and distils geometry-related features.
...
1
2
3
4
5
...