ISTA-NET++: Flexible Deep Unfolding Network for Compressive Sensing

  title={ISTA-NET++: Flexible Deep Unfolding Network for Compressive Sensing},
  author={Di You and Jingfen Xie and Jian Zhang},
  journal={2021 IEEE International Conference on Multimedia and Expo (ICME)},
While deep neural networks have achieved impressive success in image compressive sensing (CS), most of them lack flexibility when dealing with multi-ratio tasks and multi-scene images in practical applications. To tackle these challenges, we propose a novel end-to-end flexible ISTA-unfolding deep network, dubbed ISTA-Net++, with superior performance and strong flexibility. Specifically, by developing a dynamic unfolding strategy, our model enjoys the adaptability of handling CS problems with… 

Figures and Tables from this paper

D3C2-Net: Dual-Domain Deep Convolutional Coding Network for Compressive Sensing
A generalized dual-domain optimization framework is proposed, which is general for inverse imaging and integrates the merits of both image-domain and convolutional-coding-domain priors to constrain the feasible region in the solution space.
A Fast Multi-Scale Generative Adversarial Network for Image Compressed Sensing
An effective multi-scale sampling structure that contains four different kernels with varying sizes so that decompose, and sample images effectively, which is capable of capturing different levels of spatial features at multiple scales is proposed.
CSformer: Bridging Convolution and Transformer for Compressive Sensing
CSformer is proposed, a hybrid framework that integrates the advantages of leveraging both detailed spatial information from CNN and the global context provided by transformer for enhanced representation learning, and explores a progressive strategy and window-based transformer block to reduce the parameter and computational complexity.
HerosNet: Hyperspectral Explicable Reconstruction and Optimal Sampling Deep Network for Snapshot Compressive Imaging
A novel HerosNet is proposed, which includes several phases that can simulate the sensing matrix and contextually adjust the step size in the gradient descent step, and hierarchically fuse and interact the hidden states of previous phases to effectively recover current HSI frames in the proximal mapping step.
A Unifying Multi-sampling-ratio CS-MRI Framework With Two-grid-cycle Correction and Geometric Prior Distillation
A unifying deep unfolding multi-sampling-ratio CS-MRI framework that can not only compensate the lost contextual information of reconstructed image which is refined from low frequency error in geometric characteristic k-space, but also integrate the theoretical guarantee of model- based methods and the superior reconstruction performances of deep learning-based methods.
Memory-Augmented Deep Unfolding Network for Compressive Sensing
A novel DUN with persistent memory for CS is proposed, dubbed Memory-Augmented Deep Unfolding Network (MADUN), and a memory-augmented proximal mapping module (MAPMM) is designed by combining two types of memory augmentation mechanisms, namely High-throughput Short-term memory (HSM) and Cross-stage Long-term Memory (CLM).
PUERT: Probabilistic Under-sampling and Explicable Reconstruction Network for CS-MRI
A novel end-to-end Probabilistic Under-sampling and Explicable Reconstruction neTwork, dubbed PUERT, to jointly optimize the sampling pattern and the reconstruction network, which achieves state-of-the-art results in terms of both quantitative metrics and visual quality.
ICRICS: Iterative Compensation Recovery for Image Compressive Sensing
A method of iterative compensation recovery for image compressive sensing (ICRICS) is proposed by introducing closed-loop framework into traditional compresses sensing systems by introducing negative feedback structure.
Distributed Compressed Hyperspectral Sensing Imaging Incorporated Spectral Unmixing and Learning
Compressed hyperspectral imaging is a powerful technique for satellite-borne and airborne sensors that can effectively shift the complex computational burden from the resource-constrained encoding
Meta-learning based Alternating Minimization Algorithm for Non-convex Optimization
A meta-learning based alternating minimization (MLAM) method, which aims to minimize a partial of the global losses over iterations instead of carrying minimization on each sub-problem, and it tends to learn an adaptive strategy to replace the handcrafted counterpart resulting in advance on superior performance.


ISTA-Net: Interpretable Optimization-Inspired Deep Network for Image Compressive Sensing
This paper proposes a novel structured deep network, dubbed ISTA-Net, which is inspired by the Iterative Shrinkage-Thresholding Algorithm (ISTA) for optimizing a general $$ norm CS reconstruction model and develops an effective strategy to solve the proximal mapping associated with the sparsity-inducing regularizer using nonlinear transforms.
AMP-Net: Denoising-Based Deep Unfolding for Compressive Image Sensing
The proposed AMP-Net has better reconstruction accuracy than other state-of-the-art methods with high reconstruction speed and a small number of network parameters and is established by unfolding the iterative denoising process of the well-known approximate message passing algorithm.
Optimization-Inspired Compact Deep Compressive Sensing
This paper proposes a novel framework to design an OPtimization-INspired Explicable deep Network, dubbed OPINE-Net, for adaptive sampling and recovery, which allows image blocks to be sampled independently but reconstructed jointly to further enhance the performance.
Learning Non-Locally Regularized Compressed Sensing Network With Half-Quadratic Splitting
This paper proposes a non-locally regularized compressed sensing network for reconstructing image sequences, which can achieve high reconstruction quality without pre-training and is competitive to the popular methods, including sparsity prior based methods and deep learning based methods.
Convolutional Neural Networks for Noniterative Reconstruction of Compressively Sensed Images
This paper proposes a data-driven noniterative algorithm, ReconNet, which is a deep neural network learned end-to-end to map block-wise compressive measurements of the scene to the desired image blocks, and discusses how adding a fully connected layer to the existing ReconNet architecture allows for jointly learning the measurement matrix and the reconstruction algorithm in a single network.
ReconNet: Non-Iterative Reconstruction of Images from Compressively Sensed Measurements
A novel convolutional neural network architecture which takes in CS measurements of an image as input and outputs an intermediate reconstruction which is fed into an off-the-shelf denoiser to obtain the final reconstructed image, ReconNet.
Denoising Prior Driven Deep Neural Network for Image Restoration
A convolutional neural network (CNN) based denoiser that can exploit the multi-scale redundancies of natural images is proposed, which not only exploits the powerful denoising ability of DNNs, but also leverages the prior of the observation model.
Deep ADMM-Net for Compressive Sensing MRI
Experiments on MRI image reconstruction under different sampling ratios in k-space demonstrate that the proposed novel ADMM-Net algorithm significantly improves the baseline ADMM algorithm and achieves high reconstruction accuracies with fast computational speed.
Deep Unfolding Network for Image Super-Resolution
This paper proposes an end-to-end trainable unfolding network which leverages both learningbased methods and model-based methods to super-resolve blurry, noisy images for different scale factors via a single model, while maintaining the advantages of learning- based methods.
Deep networks for compressed image sensing
A deep network is proposed to recover the image, which imitates traditional compressed sensing reconstruction processes, and offers a very significant quality improvement compared against state-of-the-art ones.