Generator Surgery for Compressed Sensing
@article{SmedemarkMargulies2021GeneratorSF, title={Generator Surgery for Compressed Sensing}, author={Niklas Smedemark-Margulies and Jung Yeon Park and Max Daniels and Rose Yu and J.-W. van de Meent and Paul Hand}, journal={ArXiv}, year={2021}, volume={abs/2102.11163} }
Image recovery from compressive measurements requires a signal prior for the images being reconstructed. Recent work has explored the use of deep generative models with low latent dimension as signal priors for such problems. However, their recovery performance is limited by high representation error. We introduce a method for achieving low representation error using generators as signal priors. Using a pre-trained generator, we remove one or more initial blocks at test time and optimize over…
Figures and Tables from this paper
4 Citations
Regularized Training of Intermediate Layers for Generative Models for Inverse Problems
- Computer ScienceArXiv
- 2022
A new regularized GAN training algorithm is introduced and it is demonstrated that the learned generative model results in lower reconstruction errors across a wide range of under sampling ratios when solving compressed sensing, inpainting, and super-resolution problems.
Optimizing Intermediate Representations of Generative Models for Phase Retrieval
- Computer ScienceArXiv
- 2022
A novel variation of intermediate layer optimization (ILO) is leveraged to extend the range of the generator while still producing images consistent with the training data, and new initialization schemes are introduced that further improve the quality of the reconstruction.
Intermediate Layer Optimization for Inverse Problems using Deep Generative Models
- Computer ScienceICML
- 2021
This work proposes Intermediate Layer Optimization (ILO), a novel optimization algorithm for solving inverse problems with deep generative models that outperforms state-of-the-art methods introduced in StyleGAN-2 and PULSE for a wide range of inverse problems.
Score-Guided Intermediate Layer Optimization: Fast Langevin Mixing for Inverse Problems
- Computer ScienceICML
- 2022
The framework, SGILO, extends prior work by replacing the sparsity regularization with a generative prior in the intermediate layer by training a score-based model in the latent space of a StyleGAN-2 and using it to solve inverse problems.
References
SHOWING 1-10 OF 20 REFERENCES
Image-Adaptive GAN based Reconstruction
- Computer ScienceAAAI
- 2020
This paper suggests to mitigate the limited representation capabilities of generators by making them image-adaptive and enforcing compliance of the restoration with the observations via back-projections and empirically demonstrates the advantages of the proposed approach for image super-resolution and compressed sensing.
Image Processing Using Multi-Code GAN Prior
- Computer Science2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2020
A novel approach is proposed, called mGANprior, to incorporate the well-trained GANs as effective prior to a variety of image processing tasks, by employing multiple latent codes to generate multiple feature maps at some intermediate layer of the generator and composing them with adaptive channel importance to recover the input image.
Compressed Sensing using Generative Models
- Computer ScienceICML
- 2017
This work shows how to achieve guarantees similar to standard compressed sensing but without employing sparsity at all, and proves that, if G is L-Lipschitz, then roughly O(k log L) random Gaussian measurements suffice for an l2/l2 recovery guarantee.
Reducing the Representation Error of GAN Image Priors Using the Deep Decoder
- Computer ScienceArXiv
- 2020
A method for reducing the representation error of GAN priors by modeling images as the linear combination of a GAN prior with a Deep Decoder, an underparameterized and most importantly unlearned natural signal model similar to the Deep Image Prior.
Invertible generative models for inverse problems: mitigating representation error and dataset bias
- MathematicsICML
- 2020
It is demonstrated that invertible neural networks, which have zero representation error by design, can be effective natural signal priors at inverse problems such as denoising, compressive sensing, and inpainting.
Deep Image Prior
- Computer Science2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
- 2018
It is shown that a randomly-initialized neural network can be used as a handcrafted prior with excellent results in standard inverse problems such as denoising, super-resolution, and inpainting.
Large Scale GAN Training for High Fidelity Natural Image Synthesis
- Computer ScienceICLR
- 2019
It is found that applying orthogonal regularization to the generator renders it amenable to a simple "truncation trick," allowing fine control over the trade-off between sample fidelity and variety by reducing the variance of the Generator's input.
Inverting the Generator of a Generative Adversarial Network
- Computer ScienceIEEE Transactions on Neural Networks and Learning Systems
- 2019
This paper introduces a technique, inversion, to project data samples, specifically images, to the latent space using a pretrained GAN, and demonstrates how the proposed inversion technique may be used to quantitatively compare the performance of various GAN models trained on three image data sets.
Deep Decoder: Concise Image Representations from Untrained Non-convolutional Networks
- Computer ScienceICLR
- 2019
This paper proposes an untrained simple image model, called the deep decoder, which is a deep neural network that can generate natural images from very few weight parameters, with a simple architecture with no convolutions and fewer weight parameters than the output dimensionality.
Latent Convolutional Models
- Computer ScienceICLR
- 2019
The new model provides a strong and universal image prior for a variety of image restoration tasks such as large-hole inpainting, superresolution, and colorization and outperforms the competing approaches over a range of restoration tasks.