BIGPrior: Toward Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration

@article{ElHelou2020BIGPriorTD,
  title={BIGPrior: Toward Decoupling Learned Prior Hallucination and Data Fidelity in Image Restoration},
  author={Majed El Helou and Sabine S{\"u}sstrunk},
  journal={IEEE Transactions on Image Processing},
  year={2020},
  volume={31},
  pages={1628-1640}
}
Classic image-restoration algorithms use a variety of priors, either implicitly or explicitly. Their priors are hand-designed and their corresponding weights are heuristically assigned. Hence, deep learning methods often produce superior image restoration quality. Deep networks are, however, capable of inducing strong and hardly predictable hallucinations. Networks implicitly learn to be jointly faithful to the observed data while learning an image prior; and the separation of original data and… 
[PDF] Semantic Reader
12 Citations
Background Citations
6
Methods Citations
2

12 Citations

Learning degraded image classification with restoration data fidelity

This report explores the influence of degradation types and levels on four widely-used classification networks, and proposes a novel method leveraging a fidelity map to calibrate the image features obtained by pre-trained classification networks to mitigate the effect caused by image degradation.

RGI: robust GAN-inversion for mask-free image inpainting and unsupervised pixel-wise anomaly detection

The proposed RGI/R-RGI method unifies two important applications with state-of-the-art (SOTA) performance: mask-free semantic inpainting, where the corruptions are unknown missing regions, the restored background can be used to restore the missing content, and unsupervised pixel-wise anomaly detection.

High-resolution facial expression image restoration via adaptive total variation regularization for classroom learning environment

Target Aware Poisson-Gaussian Noise Parameters Estimation from Noisy Images

This project uses a Poisson-Gaussian noise model for the raw-images captured by the sensor, as it resembles the physical characteristics of the sensor closely and derives the theoretical maximum likelihood solution.

PoGaIN: Poisson-Gaussian Image Noise Modeling From Paired Samples

A novel, cumulant-based, approach for Poisson-Gaussian noise modeling from paired image samples is derived, which shows its improved performance over different baselines, with special emphasis on MSE, effect of outliers, image dependence, and bias.

Towards Robust Drone Vision in the Wild

This thesis proposes the first image superresolution dataset for drone vision and proposes two methods to build a robust image super-resolution network at different altitudes, which can efficiently improve the performance of super- resolution networks at varying altitudes.

Pyramid dilated convolutional neural network for image denoising

Experiments demonstrate that the proposed pyramid dilated CNN outperforms the comparative state-of-the-art denoising methods for gray and color images and can effectively deal with real-world noisy images.

TAPE: Task-Agnostic Prior Embedding for Image Restoration

A novel approach that embeds a task-agnostic prior into a transformer and shows the ability of disentangling generalized image priors from degraded images, which enjoys favorable transfer ability to unknown downstream tasks.

Image Denoising with Control over Deep Network Hallucination

This work proposes a controllable confidence-based image denoising (CCID) method, a novel framework addressing generalization and interpretability in image Denoising by exploiting reliable convolution filters and giving control and insight to the user.

A REVIEW AND ANALYSIS OF THE EXISTING LITERATURE ON MONOCHROMATIC PHOTOGRAPHY COLORIZATION USING DEEP LEARNING

This work introduces the problem, summarizes the general deep learning solutions, and discusses the experimental results obtained from open-source repositories, examining used datasets and methodologies for evaluation, data processing activities, and the infrastructure demanded by these systems.

References

SHOWING 1-10 OF 63 REFERENCES

Deep Image Prior

It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting.

Exploiting Deep Generative Prior for Versatile Image Restoration and Manipulation

This work presents an effective way to exploit the image prior captured by a generative adversarial network (GAN) trained on large-scale natural images and allows the generator to be fine-tuned on-the-fly in a progressive manner regularized by feature distance obtained by the discriminator in GAN.

Generative Image Inpainting with Contextual Attention

This work proposes a new deep generative model-based approach which can not only synthesize novel image structures but also explicitly utilize surrounding image features as references during network training to make better predictions.

Natural Image Denoising with Convolutional Networks

An approach to low-level vision is presented that combines the use of convolutional networks as an image processing architecture and an unsupervised learning procedure that synthesizes training samples from specific noise models to avoid computational difficulties in MRF approaches that arise from probabilistic learning and inference.

Beyond a Gaussian Denoiser: Residual Learning of Deep CNN for Image Denoising

This paper investigates the construction of feed-forward denoising convolutional neural networks (DnCNNs) to embrace the progress in very deep architecture, learning algorithm, and regularization method into image Denoising, and uses residual learning and batch normalization to speed up the training process as well as boost theDenoising performance.

Non-local sparse models for image restoration

Experimental results in image denoising and demosaicking tasks with synthetic and real noise show that the proposed method outperforms the state of the art, making it possible to effectively restore raw images from digital cameras at a reasonable speed and memory cost.

A content-aware image prior

An image restoration algorithm is introduced that adapts the image prior to the underlying texture, and can be adapted to both low-level local structures as well as mid-level textural characteristics, resulting in improvements in visual quality.

PULSE: Self-Supervised Photo Upsampling via Latent Space Exploration of Generative Models

This work presents a novel super-resolution algorithm, PULSE (Photo Upsampling via Latent Space Exploration), which generates high-resolution, realistic images at resolutions previously unseen in the literature, and outperforms state-of-the-art methods in perceptual quality at higher resolutions and scale factors than previously possible.

Sparsity-Based Poisson Denoising With Dictionary Learning

The scheme uses a greedy pursuit with boot-strapping-based stopping condition and dictionary learning within the denoising process, and the reconstruction performance is competitive with leading methods in high SNR and achieving state-of-the-art results in cases of low SNR.

Stochastic Frequency Masking to Improve Super-Resolution and Denoising Networks

This work presents an analysis, in the frequency domain, of degradation-kernel overfitting in super-resolution and introduces a conditional learning perspective that extends to both super- resolution and denoising and proposes a stochastic frequency masking of images used in training to regularize the networks and address the overfitting problem.
...