Deblurring via Stochastic Refinement

@article{Whang2021DeblurringVS,
  title={Deblurring via Stochastic Refinement},
  author={Jay Whang and Mauricio Delbracio and Hossein Talebi and Chitwan Saharia and Alexandros G. Dimakis and Peyman Milanfar},
  journal={2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={16272-16282}
}
  • Jay WhangM. Delbracio P. Milanfar
  • Published 5 December 2021
  • Computer Science
  • 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Image deblurring is an ill-posed problem with multiple plausible solutions for a given input image. However, most existing methods produce a deterministic estimate of the clean image and are trained to minimize pixel-level distortion. These metrics are known to be poorly correlated with human perception, and often lead to unrealistic reconstructions. We present an alternative framework for blind deblurring based on conditional diffusion models. Unlike existing techniques, we train a stochastic… 
[PDF] Semantic Reader
37 Citations
Highly Influential Citations
4
Background Citations
21
Methods Citations
15

37 Citations

Neural Image Compression with a Diffusion-Based Decoder

The resulting codec, which is called DIffuson-based Residual Augmentation Codec (DIRAC), is the first neural codec to allow smooth traversal of the rate-distortion-perception tradeoff at test time, while obtaining competitive performance with GAN-based methods in perceptual quality.

StoRM: A Diffusion-based Stochastic Regeneration Model for Speech Enhancement and Dereverberation

It is shown that the proposed approach uses the predictive model to remove the vocalizing and breathing artifacts while producing very high quality samples thanks to the diffusion model, even in adverse conditions, and enables to use lighter sampling schemes with fewer diffusion steps without sacrificing quality.

Dual Diffusion Architecture for Fisheye Image Rectification: Synthetic-to-Real Generalization

An unsupervised one-pass network that produces a plausible new condition to strengthen guidance is provided and can be regarded as an alternate scheme for fast pro-ducing reliable results without iterative inference.

A feature catcher with excellent deblurring effects

The proposed feature catcher network (FCN) outperforms state-of-the-art methods in terms of deblurring effect and also construct residual supplements for blurry images in the raw information mechanism.

Image Deblurring with Domain Generalizable Diffusion Models

This work investigates the generalization ability of icDPMs in deblurring, and proposes a simple but effective guidance to significantly alleviate artifacts, and improve the out-of-distribution performance.

DiffSDS: A language diffusion model for protein backbone inpainting under geometric conditions and constraints

Extensive experiments show that the plug-and-play ADS could transform the language model into a strong structural model without loss of simplicity, and the proposed DiffSDS outperforms previous strong baselines by a large margin on the task of protein inpainting.

Image Denoising: The Deep Learning Revolution and Beyond - A Survey Paper -

A broad view of the history of the field of image denoising and closely related topics in image processing is provided, to give a better context to recent discoveries, and to the influence of the AI revolution in this domain.

Restoring Vision in Adverse Weather Conditions with Patch-Based Denoising Diffusion Models

This work presents a novel patch-based image restoration algorithm based on denoising diffusion probabilistic models that enables size-agnostic image restoration by using a guidedDenoising process with smoothed noise estimates across overlapping patches during inference.

ResGrad: Residual Denoising Diffusion Probabilistic Models for Text to Speech

ResGrad is proposed, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model by predicting the residual between the model output and the corresponding groundtruth speech, resulting into a more lightweight model and thus a smaller real-time factor.

ADIR: Adaptive Diffusion for Image Reconstruction

This work proposes a conditional sampling scheme that exploits the prior learned by diffusion models while retaining agreement with the observations, and combines it with a novel approach for adapting pre-trained diffusion denoising networks to their input.

References

SHOWING 1-10 OF 78 REFERENCES

The Unreasonable Effectiveness of Deep Features as a Perceptual Metric

A new dataset of human perceptual similarity judgments is introduced and it is found that deep features outperform all previous metrics by large margins on this dataset, and suggests that perceptual similarity is an emergent property shared across deep visual representations.

Image quality assessment: from error visibility to structural similarity

A structural similarity index is developed and its promise is demonstrated through a set of intuitive examples, as well as comparison to both subjective ratings and state-of-the-art objective methods on a database of images compressed with JPEG and JPEG2000.

HINet: Half Instance Normalization Network for Image Restoration

A novel block is presented: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks and a simple and powerful multi-stage network named HINet is designed, which surpasses the state-of-the-art (SOTA) on various image restoration tasks.

Multi-Stage Progressive Image Restoration

This paper proposes a novel synergistic design that can optimally balance these competing goals in image restoration tasks, that progressively learns restoration functions for the degraded inputs, thereby breaking down the overall recovery process into more manageable steps.

Human-Aware Motion Deblurring

A human-aware deblurring model that disentangles the motion blur between foreground (FG) humans and background (BG) humans based on a triple-branch encoder-decoder architecture that performs favorably against the state-of-the-art motion deblurred methods, especially in capturing semantic details.

DeblurGAN-v2: Deblurring (Orders-of-Magnitude) Faster and Better

It is demonstrated that DeblurGAN-V2 has very competitive performance on several popular benchmarks, in terms of deblurring quality (both objective and subjective), as well as efficiency, and is effective for general image restoration tasks too.

U-Net: Convolutional Networks for Biomedical Image Segmentation

It is shown that such a network can be trained end-to-end from very few images and outperforms the prior best method (a sliding-window convolutional network) on the ISBI challenge for segmentation of neuronal structures in electron microscopic stacks.

DeblurGAN: Blind Motion Deblurring Using Conditional Adversarial Networks

DeblurGAN achieves state-of-the art performance both in the structural similarity measure and visual appearance and is 5 times faster than the closest competitor - Deep-Deblur.

Making a “Completely Blind” Image Quality Analyzer

This work has recently derived a blind IQA model that only makes use of measurable deviations from statistical regularities observed in natural images, without training on human-rated distorted images, and, indeed, without any exposure to distorted images.

Progressive Distillation for Fast Sampling of Diffusion Models

A method to distill a trained deterministic diffusion sampler, using many steps, into a new diffusion model that takes half as many sampling steps, and it is shown that the full progressive distillation procedure does not take more time than it takes to train the original model.
...