Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework

@article{Chen2018RobustVC,
  title={Robust Video Content Alignment and Compensation for Rain Removal in a CNN Framework},
  author={Jie Chen and Cheen-Hau Tan and Junhui Hou and Lap-Pui Chau and He Li},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={6286-6295}
}
  • Jie ChenCheen-Hau Tan He Li
  • Published 28 March 2018
  • Computer Science
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
Rain removal is important for improving the robustness of outdoor vision based systems. [] Key Method Alignment of scene contents are done at the SP level, which proves to be robust towards rain occlusion and fast camera motion. Two alignment output tensors, i.e., optimal temporal match tensor and sorted spatial-temporal match tensor, provide informative clues for rain streak location and occluded background contents to generate an intermediate derain output.
View on IEEE
arxiv.org
111 Citations
Highly Influential Citations
24
Background Citations
50
Methods Citations
55
Results Citations
3

111 Citations

Robust Video Content Alignment and Compensation for Clear Vision Through the Rain

A novel derain framework which applies superpixel (SP) segmentation to decompose the scene into depth consistent units and is not only able to suppress heavy and opaque occluding rain streaks, but also large semi-transparent regional fluctuations and distortions.

Complex scene video frames alignment and multi-frame fusion deraining with deep neural network

A two-stage method for video deraining through adjacent frames alignment and multi-frame fusion, which uses the temporal and spatial information of multiple frames to compensate and restore the details of the target deraining images, and outputs clean images.

Rain Streaks and Snowflakes Removal for Video Sequences via Motion Compensation and Matrix Completion

A three-step method to detect and remove rain streaks, even snowflakes from great majority video sequences, using motion compensation and low-rank matrix completion method, which achieves state-of-the-art performance.

Rain Streak Removal from Video Sequence using Spatiotemporal Appearance

  • M. R. IslamM. Paul
  • Computer Science
    2019 Digital Image Computing: Techniques and Applications (DICTA)
  • 2019
This paper proposes a concise model for RSRV based on the spatiotemporal appearance (STA) of rain streak in a video to exploit the rain streak appearance property and establishes that the proposed method outperforms the state-of-the-art methods significantly for both real and synthetic rains.

Spatial Attentive Single-Image Deraining With a High Quality Real Rain Dataset

A semi-automatic method that incorporates temporal priors and human supervision to generate a high-quality clean image from each input sequence of real rain images is proposed, and a novel SPatial Attentive Network (SPANet) is proposed to remove rain streaks in a local-to-global manner.

Coarse-to-fine multiscale fusion network for single image deraining

A coarse-to-fine rain removal model, starting with a rain detection network to get a coarse rain-free image, which is further refined with the Feature Attention Compensation module (FAC) to eventually create a fine rain- free image.

Self-Aligned Video Deraining with Transmission-Depth Consistency

Experimental results show that the self-alignment network developed here is effective in removing both rain streaks and rain accumulation, outperforming those of state-of-the-art methods quantitatively and qualitatively.

Rain Streak Removal for Single Image via Kernel Guided CNN

A novel rain streak removal approach using a kernel guided convolutional neural network (KGCNN), achieving the state-of-the-art performance with simple network architectures is proposed.

Let’s See Clearly: Contaminant Artifact Removal for Moving Cameras

This paper first seeks to detect attention maps that indicate the regions that need to be restored, and proposes a flow completion module to hallucinate the flow of the background scene to the attention regions degraded by the contaminants.

Survey on rain removal from videos or a single image

This work presents a comprehensive overview of existing video and single image deraining methods as well as reproduce and evaluate current state-of-the-art deraining methodologies, mainly classified into model- and deep-learning-based methods.
...

References

SHOWING 1-10 OF 36 REFERENCES

A Rain Pixel Recovery Algorithm for Videos With Highly Dynamic Scenes

The proposed algorithm is based on motion segmentation of dynamic scene and has a much better performance for rainy scenes with large motion than existing algorithms.

Removing Rain from Single Images via a Deep Detail Network

A deep detail network is proposed to directly reduce the mapping range from input to output, which makes the learning process easier and significantly outperforms state-of-the-art methods on both synthetic and real-world images in terms of both qualitative and quantitative measures.

Dynamic scene rain removal for moving cameras

A rain removal algorithm which can effectively remove rain from dynamic scenes taken from moving cameras by improving a recent state-of-the-art rain removal method by first aligning neighboring frames to a target frame before the target frame is de-rained.

Rain Streak Removal Using Layer Priors

This paper proposes an effective method that uses simple patch-based priors for both the background and rain layers that removes rain streaks better than the existing methods qualitatively and quantitatively.

Deep Joint Rain Detection and Removal from a Single Image

A recurrent rain detection and removal network that removes rain streaks and clears up the rain accumulation iteratively and progressively is proposed and a new contextualized dilated network is developed to exploit regional contextual information and to produce better representations for rain detection.

Rain Removal in Video by Combining Temporal and Chromatic Properties

A new rain removal algorithm that incorporates both temporal and chromatic properties of rain in video that can detect and remove rain streaks in both stationary and dynamic scenes taken by stationary cameras is presented.

Visual Depth Guided Color Image Rain Streaks Removal Using Sparse Coding

Compared with the state-of-the-art approaches, the proposed method is among the first to focus on the problem of single color image rain removal and achieves promising results with not only the rain component being removed more completely, but also the visual quality of restored images being improved.

Automatic Single-Image-Based Rain Streaks Removal via Image Decomposition

This paper proposes a single-image-based rain removal framework via properly formulating rain removal as an image decomposition problem based on morphological component analysis and shows that the rain component can be successfully removed from the image while preserving most original image details.

Video Deraining and Desnowing Using Temporal Correlation and Low-Rank Matrix Completion

Experimental results demonstrate that the proposed algorithm detects and removes rain or snow streaks efficiently, outperforming conventional algorithms.

SEEDS: Superpixels Extracted Via Energy-Driven Sampling

A robust and fast to evaluate energy function is defined, based on enforcing color similarity between the boundaries and the superpixel color histogram, which is able to achieve a performance comparable to the state-of-the-art, but in real-time on a single Intel i7 CPU at 2.8 GHz.