Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes

  title={Syn2Real Transfer Learning for Image Deraining Using Gaussian Processes},
  author={Rajeev Yasarla and Vishwanath A. Sindagi and Vishal M. Patel},
  journal={2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
Recent CNN-based methods for image deraining have achieved excellent performance in terms of reconstruction error as well as visual quality. However, these methods are limited in the sense that they can be trained only on fully labeled data. Due to various challenges in obtaining real world fully-labeled image deraining datasets, existing methods are trained only on synthetically generated data and hence, generalize poorly to real-world images. The use of real-world data in training image… 

Figures and Tables from this paper

Semi-Supervised Image Deraining Using Gaussian Processes

This work proposes a Gaussian Process-based semi-supervised learning framework which enables the network in learning to derain using synthetic dataset while generalizing better using unlabeled real-world images.

Exploring Overcomplete Representations for Single Image Deraining Using CNNs

This article proposes using an overcomplete convolutional network architecture which gives special attention in learning local structures by restraining the receptive field of filters and combines it with U-Net so that it does not lose out on the global structures while focusing more on low-level features, to compute the derained image.

Learning to Count in the Crowd from Limited Labeled Data

This work proposes a Gaussian Process-based iterative learning mechanism that involves estimation of pseudo-ground truth for the unlabeled data, which is then used as supervision for training the network.

From Rain Removal to Rain Generation

Single image deraining is an important yet challenging issue due to the complex and diverse rain structures in real scenes. Currently, the state-of-the-art performance on this task is achieved by

Rain to Rain: Learning Real Rain Removal Without Ground Truth

The experiments show that it is possible to train single image deraining models by using only rain images, and the idea of using the least overlapping training pairs, a method of selecting adequate training pairs that enables rain to rain training to achieve equivalent deraining performance compared to supervised training is introduced.

Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

This paper proposes a deraining algorithm that is capable of boosting the reconstruction/deraining quality without the problem of over or under-deraining and demonstrates that its approach produces better results compared to existing methods.

Progressive Rain Removal via a Recurrent Convolutional Network for Real Rain Videos

A new deep learning method for video rain removal based on recurrent neural network (RNN) architecture that focuses on the changing behaviors of rain streaks to achieve competitive results in comparison with the benchmarked and conventional approaches.

Closing the Loop: Joint Rain Generation and Removal via Disentangled Image Translation

It is argued that the rain generation and removal are the two sides of the same coin and should be tightly coupled, and a bidirectional disentangled translation network is proposed, in which each unidirectional network contains two loops of joint raingeneration and removal for both the real and synthetic rain image, respectively.

Rain Streak Removal via Dual Graph Convolutional Network

This work proposes a simple yet effective dual graph convolutional network (GCN) for single image rain removal that is easy to implement, end-to-end trainable and computationally efficient and achieves significant improvements over the recent state-of-the-art methods.

Decomposition Makes Better Rain Removal: An Improved Attention-Guided Deraining Network

An improved non-local block is designed to exploit the self-similarity of rain information by learning the holistic spatial feature correlations while reducing the calculation complexity.



Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset

A semi-automatic method that incorporates temporal priors and human supervision to generate a high-quality clean image from each input sequence of real rain images is proposed, and a novel SPatial Attentive Network (SPANet) is proposed to remove rain streaks in a local-to-global manner.

Residual-Guide Feature Fusion Network for Single Image Deraining

A novel residual-guide feature fusion network for single image deraining that progressively predicts highquality reconstruction, called ResGuideNet, is proposed and residuals generated from shallower blocks to guide deeper blocks and can obtain a coarse to fine estimation of negative residual as the blocks go deeper.

Removing Rain from Single Images via a Deep Detail Network

A deep detail network is proposed to directly reduce the mapping range from input to output, which makes the learning process easier and significantly outperforms state-of-the-art methods on both synthetic and real-world images in terms of both qualitative and quantitative measures.

Realistic Evaluation of Deep Semi-Supervised Learning Algorithms

This work creates a unified reimplemention and evaluation platform of various widely-used SSL techniques and finds that the performance of simple baselines which do not use unlabeled data is often underreported, that SSL methods differ in sensitivity to the amount of labeled and unlabeling data, and that performance can degrade substantially when the unlabelED dataset contains out-of-class examples.

Semi-Supervised Transfer Learning for Image Rain Removal

Different from traditional deep learning methods which only use supervised image pairs with/without synthesized rains, this paper further put real rainy images, without need of their clean ones, into the network training process, and trains the network to transfer to adapting the real rain pattern domain instead of only the synthesis rain domain.

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.

Uncertainty Guided Multi-Scale Residual Learning-Using a Cycle Spinning CNN for Single Image De-Raining

The proposed Uncertainty guided Multi-scale Residual Learning (UMRL) network attempts to address the single image de-raining problem by learning the rain content at different scales and using them to estimate the final de-rained output.

Clearing the Skies: A Deep Network Architecture for Single-Image Rain Removal

We introduce a deep network architecture called DerainNet for removing rain streaks from an image. Based on the deep convolutional neural network (CNN), we directly learn the mapping relationship

Domain Adaptive Faster R-CNN for Object Detection in the Wild

This work builds on the recent state-of-the-art Faster R-CNN model, and design two domain adaptation components, on image level and instance level, to reduce the domain discrepancy, based on $$-divergence theory.

Res2Net: A New Multi-Scale Backbone Architecture

This paper proposes a novel building block for CNNs, namely Res2Net, by constructing hierarchical residual-like connections within one single residual block that represents multi-scale features at a granular level and increases the range of receptive fields for each network layer.