Image Restoration with Deep Generative Models

  title={Image Restoration with Deep Generative Models},
  author={Raymond A. Yeh and Teck Yian Lim and Chen Chen and Alexander G. Schwing and Mark A. Hasegawa-Johnson and Minh N. Do},
  journal={2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  • Raymond A. Yeh, Teck Yian Lim, M. Do
  • Published 17 April 2018
  • Computer Science
  • 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
Many image restoration problems are ill-posed in nature, hence, beyond the input image, most existing methods rely on a carefully engineered image prior, which enforces some local image consistency in the recovered image. How tightly the prior assumptions are fulfilled has a big impact on the resulting task performance. To obtain more flexibility, in this work, we proposed to design the image prior in a data-driven manner. Instead of explicitly defining the prior, we learn it using deep… 

Figures and Tables from this paper

Maximum a Posteriori on a Submanifold: a General Image Restoration Method with GAN
A general method for various image restoration problems, such as denoising, deblurring, super-resolution and inpainting, using a Generative Adversarial Network (GAN) as a density estimation model.
Prior Guided GAN Based Semantic Inpainting
A data driven parametric network is learned to directly predict a matching prior for a given masked image and a recurrent net based grouped latent prior learning is proposed to extend the model for sequence reconstruction.
An Improved GAN Semantic Image Inpainting
This paper improves the semantic image inpainting and proposes a loss that generates more plausible results and achieves pixel-level photorealism, significantly outperforming state-of-the-art methods and almost independent on the size of missing area.
Fully Convolutional Pixel Adaptive Image Denoiser
A new image denoising algorithm that can learn from an offline supervised training set with a fully convolutional neural network as well as adaptively fine-tune the supervised model for each given noisy image, dubbed as Fully Convolutional Adaptive Image DEnoiser (FC-AIDE).
GAN2GAN: Generative Noise Learning for Blind Denoising with Single Noisy Images
The denoiser trained with the GAN2GAN method achieves an impressive denoising performance on both synthetic and real-world datasets for the blind Denoising setting; it almost approaches the performance of the standard discriminatively-trained or N2N-trained models that have more information than the authors', and it significantly outperforms the recent baseline for the same setting.
Image De-Quantization Using Generative Models as Priors
The goal in this work is to develop a de-quantization mechanism through a rigorous mathematical analysis which is based on the classical statistical estimation theory and includes generative modeling of the ideal image as a suitable prior information.
GAN2GAN: Generative Noise Learning for Blind Image Denoising with Single Noisy Images
The denoiser trained with the proposed GAN2GAN method achieves an impressive denoising performance, almost approaching the performance of the standard discriminatively-trained or N2N-trained models that have more information than the authors', and significantly outperforming the recent baselines for the same setting.
MRI Restoration Using Edge-Guided Adversarial Learning
This work considers the damaged image data or missing through-plane slices as image masks and proposed an edge-guided generative adversarial network to restore brain MRI images and may improve utilization of clinical 2D scans for 3D atlas generation and big-data comparative studies of brain morphometry.
Deep Learning on Image Denoising: An overview


Context Encoders: Feature Learning by Inpainting
It is found that a context encoder learns a representation that captures not just appearance but also the semantics of visual structures, and can be used for semantic inpainting tasks, either stand-alone or as initialization for non-parametric methods.
Image Super-Resolution Using Deep Convolutional Networks
We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep
Large Inpainting of Face Images With Trainlets
This letter employs a recently proposed dictionary learning framework, coined Trainlets, to design large adaptable atoms from a corpus of various datasets of face images by leveraging the online sparse dictionary learning algorithm.
Image Denoising and Inpainting with Deep Neural Networks
A novel approach to low-level vision problems that combines sparse coding and deep networks pre-trained with denoising auto-encoder (DA) is presented and can automatically remove complex patterns like superimposed text from an image, rather than simple patterns like pixels missing at random.
Image Denoising Via Learned Dictionaries and Sparse representation
  • Michael Elad, M. Aharon
  • Computer Science
    2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06)
  • 2006
This work addresses the image denoising problem, where zeromean white and homogeneous Gaussian additive noise should be removed from a given image, by defining a global image prior that forces sparsity over patches in every location in the image.
Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising
This paper investigates the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image Denoising, and uses residual learning and batch normalization to speed up the training process as well as boost theDenoising performance.
Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution
This paper proposes the Laplacian Pyramid Super-Resolution Network (LapSRN) to progressively reconstruct the sub-band residuals of high-resolution images and generates multi-scale predictions in one feed-forward pass through the progressive reconstruction, thereby facilitates resource-aware applications.
Accurate Image Super-Resolution Using Very Deep Convolutional Networks
This work presents a highly accurate single-image superresolution (SR) method using a very deep convolutional network inspired by VGG-net used for ImageNet classification and uses extremely high learning rates enabled by adjustable gradient clipping.
An Augmented Lagrangian Approach to the Constrained Optimization Formulation of Imaging Inverse Problems
This paper proposes a new efficient algorithm to handle one class of constrained problems (often known as basis pursuit denoising) tailored to image recovery applications and shows that the proposed algorithm is a strong contender for the state-of-the-art.
Generative Adversarial Nets
We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a