iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks

  title={iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks},
  author={Aman Chadha},
  journal={Computational Visual Media},
  pages={1 - 12}
  • Aman Chadha
  • Published 13 June 2020
  • Computer Science, Engineering
  • Computational Visual Media
Recently, learning-based models have enhanced the performance of single-image super-resolution (SISR). However, applying SISR successively to each video frame leads to a lack of temporal coherency. Convolutional neural networks (CNNs) outperform traditional approaches in terms of image quality metrics such as peak signal to noise ratio (PSNR) and structural similarity (SSIM). On the other hand, generative adversarial networks (GANs) offer a competitive advantage by being able to mitigate the… 
BPA-GAN: Human motion transfer using body-part-aware generative adversarial networks
A Body-Parts-Aware Generative Adversarial Network (BPA-GAN) for image-based human motion transfer that takes advantage of the human body with segmented parts instead of using the human skeleton like most of existing methods to encode the human motion information.


Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
  • C. Ledig, Lucas Theis, +6 authors W. Shi
  • Computer Science, Mathematics
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation
A novel end-to-end deep neural network that generates dynamic upsampling filters and a residual image, which are computed depending on the local spatio-temporal neighborhood of each pixel to avoid explicit motion compensation is proposed.
Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation
A novel joint motion compensation and video super-resolution algorithm that is orders of magnitude more efficient than competing methods, relying on a fast multi-resolution spatial transformer module that is end-to-end trainable is proposed.
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
This paper proposes the Laplacian Pyramid Super-Resolution Network (LapSRN) to progressively reconstruct the sub-band residuals of high-resolution images and generates multi-scale predictions in one feed-forward pass through the progressive reconstruction, thereby facilitates resource-aware applications.
Recurrent Back-Projection Network for Video Super-Resolution
We proposed a novel architecture for the problem of video super-resolution. We integrate spatial and temporal contexts from continuous video frames using a recurrent encoder-decoder module, that
Robust Video Super-Resolution with Learned Temporal Dynamics
This work proposes a temporal adaptive neural network that can adaptively determine the optimal scale of temporal dependency and reduces the complexity of motion between neighboring frames using a spatial alignment network which is much more robust and efficient than competing alignment methods.
Video Super-Resolution With Convolutional Neural Networks
This paper proposes a CNN that is trained on both the spatial and the temporal dimensions of videos to enhance their spatial resolution and shows that by using images to pretrain the model, a relatively small video database is sufficient for the training of the model to achieve and improve upon the current state-of-the-art.
Frame-Recurrent Video Super-Resolution
This work proposes an end-to-end trainable frame-recurrent video super-resolution framework that uses the previously inferred HR estimate to super-resolve the subsequent frame and demonstrates that the proposed framework is able to significantly outperform the current state of the art.
Image Super-Resolution via Deep Recursive Residual Network
This paper proposes a very deep CNN model (up to 52 convolutional layers) named Deep Recursive Residual Network (DRRN) that strives for deep yet concise networks, and recursive learning is used to control the model parameters while increasing the depth.
Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network
This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.