Path-Restore: Learning Network Path Selection for Image Restoration

@article{Yu2021PathRestoreLN,
  title={Path-Restore: Learning Network Path Selection for Image Restoration},
  author={K. Yu and Xintao Wang and Chao Dong and Xiaoou Tang and Chen Change Loy},
  journal={IEEE transactions on pattern analysis and machine intelligence},
  year={2021},
  volume={PP}
}
Very deep Convolutional Neural Networks (CNNs) have greatly improved the performance on various image restoration tasks. However, this comes at a price of increasing computational burden, hence limiting their practical usages. We observe that some corrupted image regions are inherently easier to restore than others since the distortion and content vary within an image. To leverage this, we propose Path-Restore, a multi-path CNN with a pathfinder that can dynamically select an appropriate route… 
View on PubMed
arxiv.org
41 Citations
Highly Influential Citations
5
Background Citations
21
Methods Citations
15
Results Citations
1

41 Citations

Searching for Controllable Image Restoration Networks
TLDR
A novel framework for controllable image restoration that can effectively restore multiple types and levels of degradation of a corrupted image, named TASNet, which is automatically determined by the neural architecture search algorithm, which optimizes the efficiency-accuracy trade-off of the candidate model architectures.
Image Quality Assessment Driven Reinforcement Learning for Mixed Distorted Image Restoration
TLDR
The limitations of a single deep network in dealing with multiple distortion types, then a hierarchical toolkit with distinguished powerful tools is introduced, and how an efficient representation of images combined with a reinforcement learning (RL) paradigm helps to deal with tool noise in continuous restoration is investigated.
Dynamic Residual Dense Network for Image Denoising
TLDR
The dynamic residual dense network (DRDN), a dynamic network that can selectively skip some RDBs based on the noise amount of the input image, is proposed, which can perform better than the RDN and reduces the computational cost by 40–50%.
Learning the Non-differentiable Optimization for Blind Super-Resolution
TLDR
An adaptive modulation network (AMNet) for multiple degradations SR is proposed, which is composed of the pivotal adaptive modulation layer (AMLayer) which is an efficient yet lightweight fusion layer between blur kernel and image features.
Pyramid Real Image Denoising Network
TLDR
A novel pyramid real image denoising network (PRIDNet), which contains three stages, which uses channel attention mechanism to recalibrate the channel importance of input noise and adopts a kernel selecting operation to adaptively fuse multi-scale features.
Deep Adaptive Inference Networks for Single Image Super-Resolution
TLDR
The AdaDSR involves an SISR model as backbone and a lightweight adapter module which takes image features and resource constraint as input and predicts a map of local network depth, which can be flexibly tuned to meet a range of efficiency constraints.
R3L: Connecting Deep Reinforcement Learning To Recurrent Neural Networks For Image Denoising Via Residual Recovery
TLDR
It is demonstrated that the proposed R3L has better generalizability and robustness in image denoising when the estimated noise level varies, comparing to its counterparts using deterministic training, as well as various state-of-the-art image Denoising algorithms.
A two-stage modular blind denoising algorithm based on real scene
TLDR
A relatively simple network structure to remove real image noise, which includes the noise level estimation stage using the channel attention mechanism, and the non-blind noise reduction stage uses the micro-branch structure the authors designed.
Restoring Spatially-Heterogeneous Distortions using Mixture of Experts Network
TLDR
A spatially-heterogeneous distortion dataset in which multiple corruptions are applied to the different locations of each image is introduced and a mixture of experts network is proposed to effectively restore a multi-distortion image.
Interactive Multi-dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration
TLDR
A new problem setup, called multi- dimension (MD) modulation, which aims at modulating output effects across multiple degradation types and levels, and a deep architecture - CResMD with newly introduced controllable residual connections for multi-dimension modulation is presented.
...
...

References

SHOWING 1-10 OF 66 REFERENCES
Non-local Color Image Denoising with Convolutional Neural Networks
  • Stamatios Lefkimmiatis
  • Computer Science
    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2017
TLDR
This work proposes a novel deep network architecture for grayscale and color image denoising that is based on a non-local image model and highlights a direct link of the proposed non- local models to convolutional neural networks.
Pyramid Real Image Denoising Network
TLDR
A novel pyramid real image denoising network (PRIDNet), which contains three stages, which uses channel attention mechanism to recalibrate the channel importance of input noise and adopts a kernel selecting operation to adaptively fuse multi-scale features.
Image Super-Resolution Using Deep Convolutional Networks
We propose a deep learning method for single image super-resolution (SR). Our method directly learns an end-to-end mapping between the low/high-resolution images. The mapping is represented as a deep…
BlockDrop: Dynamic Inference Paths in Residual Networks
TLDR
BlockDrop, an approach that learns to dynamically choose which layers of a deep network to execute during inference so as to best reduce total computation without degrading prediction accuracy, is introduced.
Image Super-Resolution via Deep Recursive Residual Network
TLDR
This paper proposes a very deep CNN model (up to 52 convolutional layers) named Deep Recursive Residual Network (DRRN) that strives for deep yet concise networks, and recursive learning is used to control the model parameters while increasing the depth.
FFDNet: Toward a Fast and Flexible Solution for CNN-Based Image Denoising
TLDR
The proposed FFDNet works on downsampled sub-images, achieving a good trade-off between inference speed and denoising performance, and enjoys several desirable properties, including the ability to handle a wide range of noise levels effectively with a single network.
Enhanced Deep Residual Networks for Single Image Super-Resolution
TLDR
This paper develops an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods, and proposes a new multi-scale deepsuper-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
TLDR
This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.
Toward Convolutional Blind Denoising of Real Photographs
TLDR
A convolutional blind denoising network (CBDNet) with more realistic noise model and real-world noisy-clean image pairs and a noise estimation subnetwork with asymmetric learning to suppress under-estimation of noise level is embedded into CBDNet.
Natural Image Denoising with Convolutional Networks
TLDR
An approach to low-level vision is presented that combines the use of convolutional networks as an image processing architecture and an unsupervised learning procedure that synthesizes training samples from specific noise models to avoid computational difficulties in MRF approaches that arise from probabilistic learning and inference.
...
...