SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation

@article{Son2021SRWarpGI,
  title={SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation},
  author={Sanghyun Son and Kyoung Mu Lee},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={7778-7787}
}
  • Sanghyun Son, Kyoung Mu Lee
  • Published 21 April 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution (SR). However, conventional methods still resort to some predetermined integer scaling factors, e.g., ×2 or ×4. Thus, they are difficult to be applied when arbitrary target resolutions are required. Recent approaches ex-tend the scope to real-valued upsampling factors, even with varying aspect ratios to handle the limitation. In this pa-per, we propose the SRWarp… 

Figures and Tables from this paper

Learning Local Implicit Fourier Representation for Image Warping

TLDR
This paper proposes a local texture estimator for image warping (LTEW) followed by an implicit neural representation to deform images into continuous shapes and outperforms existing warping methods for asymmetric-scale SR and homography transform.

Single Image Super-Resolution with Arbitrary Magnification Based on High-Frequency Attention Network

TLDR
Experimental results show that the proposed method has excellent restoration performance, both quantitatively and qualitatively, compared to the existing arbitrary magnification super-resolution methods.

CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution

TLDR
A novel CADyQ framework for SR networks that allocates optimal bits to local regions and layers adaptively based on the local contents of an input image to achieve high average bit-reduction with less accuracy loss is proposed.

Attentive Fine-Grained Structured Sparsity for Image Restoration

TLDR
This work proposes a novel pruning method that determines the pruning ratio for N : M structured sparsity at each layer of an image restoration network, which outperforms previous pruning methods significantly.

SphereSR: 360° Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

TLDR
Experiments on various benchmark datasets show that the proposed SphereSR significantly surpasses existing methods in terms of performance, and flexibly reconstructs an HR image given an arbitrary projection type.

Local Texture Estimator for Implicit Representation Function

TLDR
A Local Texture Estimator (LTE), a dominant-frequency estimator for natural images, enabling an implicit function to capture details while reconstructing images in a continuous manner, and shows that an LTE-based neural function achieves favorable performance against existing deep SR methods within an arbitrary-scale factor.

References

SHOWING 1-10 OF 60 REFERENCES

NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study

  • E. AgustssonR. Timofte
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2017
TLDR
It is concluded that the NTIRE 2017 challenge pushes the state-of-the-art in single-image super-resolution, reaching the best results to date on the popular Set5, Set14, B100, Urban100 datasets and on the authors' newly proposed DIV2K.

Learning for Scale-Arbitrary Super-Resolution from Scale-Specific Networks

TLDR
This paper proposes a plug-in module for existing SR networks to perform scale-arbitrary SR, which consists of multiple scale- aware feature adaption blocks and a scale-aware upsampling layer, and introduces a Scale-aware knowledge transfer paradigm to transfer knowledge from scale-specific networks to the scale-Arbitrary network.

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

TLDR
This work thoroughly study three key components of SRGAN – network architecture, adversarial loss and perceptual loss, and improves each of them to derive an Enhanced SRGAN (ESRGAN), which achieves consistently better visual quality with more realistic and natural textures than SRGAN.

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

TLDR
This work presents a highly accurate single-image superresolution (SR) method using a very deep convolutional network inspired by VGG-net used for ImageNet classification and uses extremely high learning rates enabled by adjustable gradient clipping.

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

TLDR
A database containing 'ground truth' segmentations produced by humans for images of a wide variety of natural scenes is presented and an error measure is defined which quantifies the consistency between segmentations of differing granularities.

Enhanced Deep Residual Networks for Single Image Super-Resolution

TLDR
This paper develops an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods, and proposes a new multi-scale deepsuper-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model.

Meta-SR: A Magnification-Arbitrary Network for Super-Resolution

TLDR
This work proposes a novel method called Meta-SR to firstly solvesuper-resolution of arbitrary scale factor (including non-integer scale factors) with a single model and shows the superi-ority of the Meta-Upscale.

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

TLDR
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

TLDR
This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.

Image Super-Resolution Using Deep Convolutional Networks

We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep
...