Channel Attention Is All You Need for Video Frame Interpolation

@inproceedings{Choi2020ChannelAI,
  title={Channel Attention Is All You Need for Video Frame Interpolation},
  author={Myungsub Choi and Heewon Kim and Bohyung Han and N. Xu and Kyoung Mu Lee},
  booktitle={AAAI},
  year={2020}
}
Prevailing video frame interpolation techniques rely heavily on optical flow estimation and require additional model complexity and computational cost; it is also susceptible to error propagation in challenging scenarios with large motion and heavy occlusion. To alleviate the limitation, we propose a simple but effective deep neural network for video frame interpolation, which is end-to-end trainable and is free from a motion estimation network component. Our algorithm employs a special feature… Expand
Scene-Adaptive Video Frame Interpolation via Meta-Learning
TLDR
This work shows the benefits of 'test-time adaptation' through simple fine-tuning of a network, then improves its efficiency by incorporating meta-learning, and obtains significant performance gains with only a single gradient update without any additional parameters. Expand
Learning Deep Video Stabilization without Optical Flow
TLDR
This work presents an iterative frame interpolation strategy to generate a novel dataset that is diverse enough to formulate video stabilization as a supervised learning problem unassisted by optical flow and provides qualitatively and quantitatively better results than those generated through state-of-the-art video stabilization methods. Expand
Flow-aware synthesis: A generic motion model for video frame interpolation
TLDR
Qualitative and quantitative experimental results show that the generic adaptive flow prediction module presented can produce high-quality results and outperforms the existing state-of-the-art methods on popular public datasets. Expand
EA-Net: Edge-Aware Network for Flow-based Video Frame Interpolation
TLDR
The proposed Edge-Aware Network (EA-Net) integrates the edge information into the frame interpolation task and is adopted to conduct the adversarial training strategy, so as to enhance the reality and clarity of synthesized frames. Expand
Enhanced Quadratic Video Interpolation
TLDR
This work improves the performance of QVI from three facets and proposes an enhanced quadratic video interpolation (EQVI) model that won the first place in the AIM2020 Video Temporal Super-Resolution Challenge and devise a novel multi-scale fusion network (MS-Fusion) which can be regarded as a learnable augmentation process. Expand
XVFI: eXtreme Video Frame Interpolation
TLDR
An extreme VFI network is proposed, called XVFI-Net, that first handles the VFI for 4K videos with large motion that performs comparably on the previous lower resolution benchmark dataset, which shows a robustness of the algorithm. Expand
Texture-aware Video Frame Interpolation
TLDR
This study shows that video texture has significant impact on the performance of frame interpolation models and it is beneficial to have separate models specifically adapted to these texture classes, instead of training a single model that tries to learn generic motion. Expand
ALANET: Adaptive Latent Attention Network for Joint Video Deblurring and Interpolation
TLDR
A novel architecture, Adaptive Latent Attention Network (ALANET), is introduced, which synthesizes sharp high frame-rate videos with no prior knowledge of input frames being blurry or not, thereby performing the task of both deblurring and interpolation. Expand
Revisiting Adaptive Convolutions for Video Frame Interpolation
TLDR
This work shows, somewhat surprisingly, that it is possible to achieve near state-of-the-art results with an older, simpler approach, namely adaptive separable convolutions, by a subtle set of low level improvements. Expand
Asymmetric Bilateral Motion Estimation for Video Frame Interpolation
We propose a novel video frame interpolation algorithm based on asymmetric bilateral motion estimation (ABME), which synthesizes an intermediate frame between two input frames. First, we predictExpand
...
1
2
3
...

References

SHOWING 1-10 OF 48 REFERENCES
Depth-Aware Video Frame Interpolation
TLDR
A video frame interpolation method which explicitly detects the occlusion by exploring the depth information, and develops a depth-aware flow projection layer to synthesize intermediate flows that preferably sample closer objects than farther ones. Expand
Deep Video Frame Interpolation Using Cyclic Frame Generation
TLDR
A new loss term, the cycle consistency loss, which can better utilize the training data to not only enhance the interpolation results, but also maintain the performance better with less training data is introduced. Expand
Super SloMo: High Quality Estimation of Multiple Intermediate Frames for Video Interpolation
TLDR
This work proposes an end-to-end convolutional neural network for variable-length multi-frame video interpolation, where the motion interpretation and occlusion reasoning are jointly modeled. Expand
Video Frame Interpolation via Adaptive Convolution
TLDR
This paper presents a robust video frame interpolation method that considers pixel synthesis for the interpolated frame as local convolution over two input frames and employs a deep fully convolutional neural network to estimate a spatially-adaptive convolution kernel for each pixel. Expand
MEMC-Net: Motion Estimation and Motion Compensation Driven Neural Network for Video Interpolation and Enhancement
TLDR
A novel adaptive warping layer is developed to integrate both optical flow and interpolation kernels to synthesize target frame pixels and is fully differentiable such that both the flow and kernel estimation networks can be optimized jointly. Expand
PhaseNet for Video Frame Interpolation
TLDR
This work proposes a new approach, PhaseNet, that is designed to robustly handle challenging scenarios while also coping with larger motion, and shows that this is superior to the hand-crafted heuristics previously used in phase-based methods and compares favorably to recent deep learning based approaches for video frame interpolation on challenging datasets. Expand
Video Frame Synthesis Using Deep Voxel Flow
TLDR
This work addresses the problem of synthesizing new video frames in an existing video, either in-between existing frames (interpolation), or subsequent to them (extrapolation), by training a deep network that learns to synthesize video frames by flowing pixel values from existing ones, which is called deep voxel flow. Expand
Optical Flow Guided TV-L1 Video Interpolation and Restoration
TLDR
This paper uses a variant of the TV-L1 denoising algorithm that operates on image sequences in a space-time volume that is modified to take the pixels' movement into account and utilizes optical flow to estimate the velocity vectors between consecutive frames. Expand
Video Enhancement with Task-Oriented Flow
TLDR
T task-oriented flow (TOFlow), a motion representation learned in a self-supervised, task-specific manner, is proposed, which outperforms traditional optical flow on standard benchmarks as well as the Vimeo-90K dataset in three video processing tasks. Expand
Video Frame Interpolation via Adaptive Separable Convolution
TLDR
This paper develops a deep fully convolutional neural network that takes two input frames and estimates pairs of 1D kernels for all pixels simultaneously, which allows for the incorporation of perceptual loss to train the neural network to produce visually pleasing frames. Expand
...
1
2
3
4
5
...