• Corpus ID: 240354701

Render In-between: Motion Guided Video Synthesis for Action Interpolation

@inproceedings{Ho2021RenderIM,
  title={Render In-between: Motion Guided Video Synthesis for Action Interpolation},
  author={Hsuan-I Ho and Xu Chen and Jie Song and Otmar Hilliges},
  booktitle={British Machine Vision Conference},
  year={2021}
}
Upsampling videos of human activity is an interesting yet challenging task with many potential applications ranging from gaming to entertainment and sports broad-casting. The main difficulty in synthesizing video frames in this setting stems from the highly complex and non-linear nature of human motion and the complex appearance and texture of the body. We propose to address these issues in a motion-guided frame-upsampling framework that is capable of producing realistic human motion and… 

Figures and Tables from this paper

DeciWatch: A Simple Baseline for 10x Efficient 2D and 3D Pose Estimation

This work presents results of efficient labeling, generalization ability, and qualitative comparison results of DeciWatch on different sampling ratios, and discusses some failure cases in this method to motivate further research.

References

SHOWING 1-10 OF 95 REFERENCES

Learning character-agnostic motion for motion retargeting in 2D

This paper presents a new method for retargeting video-captured motion between different human performers, without the need to explicitly reconstruct 3D poses and/or camera parameters, and demonstrates that this framework can be used to robustly extract human motion from videos, bypassing 3D reconstruction, and outperforming existing retargeted methods, when applied to videos in-the-wild.

Deep Video Generation, Prediction and Completion of Human Action Sequences

This paper proposes a general, two-stage deep framework to generate human action videos with no constraints or arbitrary number of constraints, which uniformly address the three problems: video generation given no input frames, video prediction given the first few frames, and video completionGiven the first and last frames.

Video Frame Synthesis Using Deep Voxel Flow

This work addresses the problem of synthesizing new video frames in an existing video, either in-between existing frames (interpolation), or subsequent to them (extrapolation), by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which is called deep voxel flow.

Few-shot Video-to-Video Synthesis

A few-shot vid2vid framework is proposed, which learns to synthesize videos of previously unseen subjects or scenes by leveraging few example images of the target at test time by utilizing a novel network weight generation module utilizing an attention mechanism.

Depth-Aware Video Frame Interpolation

A video frame interpolation method which explicitly detects the occlusion by exploring the depth information, and develops a depth-aware flow projection layer to synthesize intermediate flows that preferably sample closer objects than farther ones.

Human Motion Prediction via Spatio-Temporal Inpainting

This work argues that the L2 metric, considered so far by most approaches, fails to capture the actual distribution of long-term human motion, and proposes two alternative metrics, based on the distribution of frequencies, that are able to capture more realistic motion patterns.

Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation

This work proposes an end-to-end convolutional neural network for variable-length multi-frame video interpolation, where the motion interpretation and occlusion reasoning are jointly modeled.

Deep Video Frame Interpolation Using Cyclic Frame Generation

A new loss term, the cycle consistency loss, which can better utilize the training data to not only enhance the interpolation results, but also maintain the performance better with less training data is introduced.

DIFRINT: Deep Iterative Frame Interpolation for Full-Frame Video Stabilization

  • Jinsoo ChoiI. Kweon
  • Computer Science
    2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
  • 2019
This work proposes an unsupervised deep approach to full-frame video stabilization which can generate video frames without cropping and low distortion, and utilizes frame interpolation techniques to generate in between frames, leading to reduced inter-frame jitter.

Channel Attention Is All You Need for Video Frame Interpolation

A simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component, and achieves outstanding performance compared to the existing models with a component for optical flow computation.
...