SDAN: Squared Deformable Alignment Network for Learning Misaligned Optical Zoom

  title={SDAN: Squared Deformable Alignment Network for Learning Misaligned Optical Zoom},
  author={Kangfu Mei and Shenglong Ye and Rui Huang},
Deep Neural Network (DNN) based super-resolution algorithms have greatly improved the quality of the generated images. However, these algorithms often yield significant artifacts when dealing with real-world super-resolution problems due to the difficulty in learning misaligned optical zoom. In this paper, we introduce a Squared Deformable Alignment Network (SDAN) to address this issue. Our network learns squared per-point offsets for convolutional kernels, and then aligns features in corrected… Expand

Figures and Tables from this paper


Deformable Convolutional Networks
This work introduces two new modules to enhance the transformation modeling capability of CNNs, namely, deformable convolution and deformable RoI pooling, based on the idea of augmenting the spatial sampling locations in the modules with additional offsets and learning the offsets from the target tasks, without additional supervision. Expand
Understanding Deformable Alignment in Video Super-Resolution
It is shown that deformable convolution can be decomposed into a combination of spatial warping and convolution, which reveals the commonality of deformable alignment and flow-based alignment in formulation, but with a key difference in their offset diversity. Expand
EDVR: Video Restoration With Enhanced Deformable Convolutional Networks
This work proposes a novel Video Restoration framework with Enhanced Deformable convolutions, termed EDVR, and proposes a Temporal and Spatial Attention (TSA) fusion module, in which attention is applied both temporally and spatially, so as to emphasize important features for subsequent restoration. Expand
Image Super-Resolution Using Deep Convolutional Networks
We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deepExpand
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
  • C. Ledig, Lucas Theis, +6 authors W. Shi
  • Computer Science, Mathematics
  • 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss. Expand
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
This work thoroughly study three key components of SRGAN – network architecture, adversarial loss and perceptual loss, and improves each of them to derive an Enhanced SRGAN (ESRGAN), which achieves consistently better visual quality with more realistic and natural textures than SRGAN. Expand
Enhanced Deep Residual Networks for Single Image Super-Resolution
This paper develops an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods, and proposes a new multi-scale deepsuper-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model. Expand
Multi-scale Residual Network for Image Super-Resolution
A novel multi-scale residual network (MSRN) to fully exploit the image features, which outperform most of the state-of-the-art methods. Expand
Zoom to Learn, Learn to Zoom
This paper shows that when applying machine learning to digital zoom, it is beneficial to operate on real, RAW sensor data, and shows how to obtain such ground-truth data via optical zoom and contribute a dataset, SR-RAW, for real-world computational zoom. Expand
Deformable ConvNets V2: More Deformable, Better Results
This work presents a reformulation of Deformable Convolutional Networks that improves its ability to focus on pertinent image regions, through increased modeling power and stronger training, and guides network training via a proposed feature mimicking scheme that helps the network to learn features that reflect the object focus and classification power of R-CNN features. Expand