SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation

  title={SRWarp: Generalized Image Super-Resolution under Arbitrary Transformation},
  author={Sanghyun Son and Kyoung Mu Lee},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  • Sanghyun SonKyoung Mu Lee
  • Published 21 April 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Deep CNNs have achieved significant successes in image processing and its applications, including single image super-resolution (SR). However, conventional methods still resort to some predetermined integer scaling factors, e.g., ×2 or ×4. Thus, they are difficult to be applied when arbitrary target resolutions are required. Recent approaches ex-tend the scope to real-valued upsampling factors, even with varying aspect ratios to handle the limitation. In this pa-per, we propose the SRWarp… 

Figures and Tables from this paper

Learning Local Implicit Fourier Representation for Image Warping

This paper proposes a local texture estimator for image warping (LTEW) followed by an implicit neural representation to deform images into continuous shapes and outperforms existing warping methods for asymmetric-scale SR and homography transform.

Single Image Super-Resolution with Arbitrary Magnification Based on High-Frequency Attention Network

Experimental results show that the proposed method has excellent restoration performance, both quantitatively and qualitatively, compared to the existing arbitrary magnification super-resolution methods.

CADyQ: Content-Aware Dynamic Quantization for Image Super-Resolution

A novel CADyQ framework for SR networks that allocates optimal bits to local regions and layers adaptively based on the local contents of an input image to achieve high average bit-reduction with less accuracy loss is proposed.

Dynamic Depth-Aware Network for Endoscopy Super-Resolution

A dynamic depth-aware network for endoscopy super-resolution is proposed, which represents the first effort to comprehensively integrate the depth information to the SR task for endoscopic images and adaptively super-resolve the regions under different depth levels.

Attentive Fine-Grained Structured Sparsity for Image Restoration

This work proposes a novel pruning method that determines the pruning ratio for N:M structured sparsity at each layer of an image restoration network and demonstrates the efficacy of this method which outperforms previous pruning methods significantly.

Multi-scale generative adversarial network for image super-resolution

A multi-scale generative adversarial network used to identify the authenticity of the input image after refactoring and performs better in two indexes PSNR and SSIM of the super-resolution task.

SphereSR: $360^{\circ}$ Image Super-Resolution with Arbitrary Projection via Continuous Spherical Image Representation

The proposed SphereSR proposes a feature extraction module that represents the spherical data based on an icosahedron and that efficiently extracts features on the spherical surface, and a spherical local implicit image function (SLIIF) to predict RGB values at the spherical coordinates.

Local Texture Estimator for Implicit Representation Function

  • Jaewon LeeK. Jin
  • Computer Science
    2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2022
A Local Texture Estimator (LTE), a dominant-frequency estimator for natural images, enabling an implicit function to capture fine details while reconstructing images in a continuous manner, and shows that an LTE-based neural function achieves favorable performance against existing deep SR methods within an arbitrary-scale factor.



NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study

  • E. AgustssonR. Timofte
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
  • 2017
It is concluded that the NTIRE 2017 challenge pushes the state-of-the-art in single-image super-resolution, reaching the best results to date on the popular Set5, Set14, B100, Urban100 datasets and on the authors' newly proposed DIV2K.

Learning for Scale-Arbitrary Super-Resolution from Scale-Specific Networks

This paper proposes a plug-in module for existing SR networks to perform scale-arbitrary SR, which consists of multiple scale- aware feature adaption blocks and a scale-aware upsampling layer, and introduces a Scale-aware knowledge transfer paradigm to transfer knowledge from scale-specific networks to the scale-Arbitrary network.

ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

This work thoroughly study three key components of SRGAN – network architecture, adversarial loss and perceptual loss, and improves each of them to derive an Enhanced SRGAN (ESRGAN), which achieves consistently better visual quality with more realistic and natural textures than SRGAN.

Accurate Image Super-Resolution Using Very Deep Convolutional Networks

This work presents a highly accurate single-image superresolution (SR) method using a very deep convolutional network inspired by VGG-net used for ImageNet classification and uses extremely high learning rates enabled by adjustable gradient clipping.

A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics

A database containing 'ground truth' segmentations produced by humans for images of a wide variety of natural scenes is presented and an error measure is defined which quantifies the consistency between segmentations of differing granularities.

Enhanced Deep Residual Networks for Single Image Super-Resolution

This paper develops an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods, and proposes a new multi-scale deepsuper-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model.

Meta-SR: A Magnification-Arbitrary Network for Super-Resolution

This work proposes a novel method called Meta-SR to firstly solvesuper-resolution of arbitrary scale factor (including non-integer scale factors) with a single model and shows the superi-ority of the Meta-Upscale.

Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network

SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.

Real-Time Single Image and Video Super-Resolution Using an Efficient Sub-Pixel Convolutional Neural Network

This paper presents the first convolutional neural network capable of real-time SR of 1080p videos on a single K2 GPU and introduces an efficient sub-pixel convolution layer which learns an array of upscaling filters to upscale the final LR feature maps into the HR output.

Image Super-Resolution Using Deep Convolutional Networks

We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep