iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks

@article{Chadha2020iSeeBetterSV,
  title={iSeeBetter: Spatio-temporal video super-resolution using recurrent generative back-projection networks},
  author={Aman Chadha},
  journal={Computational Visual Media},
  year={2020},
  pages={1 - 12}
}
  • Aman Chadha
  • Published 13 June 2020
  • Computer Science
  • Computational Visual Media
Recently, learning-based models have enhanced the performance of single-image super-resolution (SISR). However, applying SISR successively to each video frame leads to a lack of temporal coherency. Convolutional neural networks (CNNs) outperform traditional approaches in terms of image quality metrics such as peak signal to noise ratio (PSNR) and structural similarity (SSIM). On the other hand, generative adversarial networks (GANs) offer a competitive advantage by being able to mitigate the… 

INR-V: A Continuous Representation Space for Videos

  • Computer Science
  • 2022
This work proposes INR-V, a video representation network that learns a continuous latent space directly for videos and analyzes its several video-based properties, demonstrating the potential of a continuous representation space for videos.

INR-V: A Continuous Representation Space for Video-based Generative Tasks

The space learned by INR-V on diverse generative tasks such as video interpolation, novel video generation, video inversion, and video inpainting against the existing baselines is evaluated and significantly outperforms.

Towards True Detail Restoration for Super-Resolution: A Benchmark and a Quality Metric

A benchmark based on the authors' own video dataset, which contains complex patterns that SR models generally fail to cor-rectly restore, is developed and an objective assessment metric is developed that outperforms other quality metrics by correlation with subjective scores for this task.

Video super-resolution for single-photon LIDAR

Synthetic depth sequences are used to train a 3D Convolutional Neural Network (CNN) for denoising and upscaling ( × 4) depth data, making the approach suitable for low-latency imaging, as required for obstacle avoidance.

Conditional Generative Adversarial Networks with Total Variation and Color Correction for Generating Indonesian Face Photo from Sketch

This study applies a conditional generative adversarial network (cGAN) to convert a face sketch image into a color face photo with an additional Total Variation (TV) term in the loss function to improve the visual quality of the resulting image.

Research on Super-Resolution Relationship Extraction and Reconstruction Methods for Images Based on Multimodal Graph Convolutional Networks

  • Jie Xiao
  • Computer Science
    Mathematical Problems in Engineering
  • 2022
A multiscale GAN-based image super-resolution reconstruction algorithm aimed at the problems of detail loss or blurring in the reconstruction of detail-rich images by SRGAN, which incorporates the idea of the Laplace pyramid to complete the task of multiscales reconstruction of images through staged reconstruction.

Robust Extraction and Super-Resolution of Low-Resolution Flying Airplane From Satellite Video

A novel constructive model is proposed to model the airplane of low resolution for more complete extraction, and a new reflective symmetry shape prior is integrated into the super-resolution process to obtain the higher resolution result.

Data-driven personalisation of television content: a survey

This survey considers the vision of TV broadcasting where content is personalised and personalisation is data-driven, looks at the AI and data technologies making this possible and surveys the

Reconstruction Methods for Images Based on Multimodal Graph Convolutional Networks

A multiscale GAN-based image super-resolution reconstruction algorithm aimed at the problems of detail loss or blurring in the reconstruction of detail-rich images by SRGAN, which integrates the idea of the Laplace pyramid to complete the task of multiscales reconstruction of images through staged reconstruction.

References

SHOWING 1-10 OF 51 REFERENCES

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.

Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation

A novel end-to-end deep neural network that generates dynamic upsampling filters and a residual image, which are computed depending on the local spatio-temporal neighborhood of each pixel to avoid explicit motion compensation is proposed.

Real-Time Video Super-Resolution with Spatio-Temporal Networks and Motion Compensation

A novel joint motion compensation and video super-resolution algorithm that is orders of magnitude more efficient than competing methods, relying on a fast multi-resolution spatial transformer module that is end-to-end trainable is proposed.

Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution

This paper proposes the Laplacian Pyramid Super-Resolution Network (LapSRN) to progressively reconstruct the sub-band residuals of high-resolution images and generates multi-scale predictions in one feed-forward pass through the progressive reconstruction, thereby facilitates resource-aware applications.

Recurrent Back-Projection Network for Video Super-Resolution

We proposed a novel architecture for the problem of video super-resolution. We integrate spatial and temporal contexts from continuous video frames using a recurrent encoder-decoder module, that

Robust Video Super-Resolution with Learned Temporal Dynamics

This work proposes a temporal adaptive neural network that can adaptively determine the optimal scale of temporal dependency and reduces the complexity of motion between neighboring frames using a spatial alignment network which is much more robust and efficient than competing alignment methods.

Video Super-Resolution With Convolutional Neural Networks

This paper proposes a CNN that is trained on both the spatial and the temporal dimensions of videos to enhance their spatial resolution and shows that by using images to pretrain the model, a relatively small video database is sufficient for the training of the model to achieve and improve upon the current state-of-the-art.

Frame-Recurrent Video Super-Resolution

This work proposes an end-to-end trainable frame-recurrent video super-resolution framework that uses the previously inferred HR estimate to super-resolve the subsequent frame and demonstrates that the proposed framework is able to significantly outperform the current state of the art.

Image Super-Resolution via Deep Recursive Residual Network

This paper proposes a very deep CNN model (up to 52 convolutional layers) named Deep Recursive Residual Network (DRRN) that strives for deep yet concise networks, and recursive learning is used to control the model parameters while increasing the depth.

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.
...