Single Image Reflection Separation with Perceptual Losses

@article{Zhang2018SingleIR,
  title={Single Image Reflection Separation with Perceptual Losses},
  author={Xuaner Cecilia Zhang and Ren Ng and Qifeng Chen},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={4786-4794}
}
  • X. Zhang, Ren Ng, Qifeng Chen
  • Published 1 June 2018
  • Computer Science
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
We present an approach to separating reflection from a single image. [] Key Method We also propose a novel exclusion loss that enforces pixel-level layer separation. We create a dataset of real-world images with reflection and corresponding ground-truth transmission layers for quantitative evaluation and model training. We validate our method through comprehensive quantitative experiments and show that our approach outperforms state-of-the-art reflection removal methods in PSNR, SSIM, and perceptual user…

Figures and Tables from this paper

Single Image Reflection Removal Based on GAN With Gradient Constraint
TLDR
This paper proposes a novel gradient constraint loss in order to separate the background layer and the reflection layer clearly and outperforms state-of-the-art methods in PSNR and SSIM, especially in real world images.
Unsupervised Single-Image Reflection Separation Using Perceptual Deep Image Priors
TLDR
A novel unsupervised framework for single-image reflection separation is proposed, which designs a new architecture of the network to embed semantic features extracted from a pre-trained deep classification network, which gives more meaningful separation similar to human perception.
ReflectNet - A Generative Adversarial Method for Single Image Reflection Suppression
TLDR
A single image reflection removal method based on context understanding modules and adversarial training to efficiently restore the transmission layer without reflection is proposed and outperforms state-of-the-art methods in terms of PSNR and SSIM on the SIR benchmark dataset.
Blind Single Image Reflection Suppression for Face Images using Deep Generative Priors
TLDR
This paper uses a pre-trained deep face-generative model for reflection supression from face images and designs an optimization scheme that effectively leverages the deep generative model and leads to a constrained solution space.
GAN-based single-image reflectance removal using depth of field guidance
TLDR
A Generative Adversarial Networks guided by using Depth of Field (DoF) is proposed, which demonstrates that the proposed network convincingly eliminates the reflection and produce sufficient transmitted layers as compared to the previous methods.
Image Reflection Removal Using the Wasserstein Generative Adversarial Network
  • Tingtian Li, D. Lun
  • Computer Science
    ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)
  • 2019
TLDR
Experimental results show that the proposed deep-learning based method to allow fast removal of reflection removal can achieve state-of-the-art performance, and is significantly faster than the traditional methods due to the use of the deep learning methods.
Single Image Reflection Removal With Physically-Based Training Images
TLDR
This paper utilizes existing RGBD/RGB images to estimate meshes, then physically simulate the light transportation between meshes, glass, and lens with path tracing to synthesize training data, which successfully reproduce the spatially variant anisotropic visual effect of glass reflection.
Single Image Reflection Removal Exploiting Misaligned Training Data and Network Enhancements
TLDR
Experimental results collectively show that the method outperforms the state-of-the-art with aligned data, and that significant improvements are possible when using additional misaligned data.
A Model-Guided Unfolding Network for Single Image Reflection Removal
TLDR
An explainable approach for SIRR named model-guided unfolding network(MoG-SIRR), which is unfolded from the proposed reflection removal model with non-local autoregressive prior and dereflection prior, which achieves superior performance for single image reflection removal.
Single Image Reflection Removal via Attention Model and SN-GAN
TLDR
An improved generative adversarial network that considers the reflection layer as a spatial mask with space-relevant reflection intensity information, which can be used to elevate the performance of the discriminator and design a novel SNGAN structure that outperforms several state-of-the-art networks.
...
...

References

SHOWING 1-10 OF 36 REFERENCES
A Generic Deep Architecture for Single Image Reflection Removal and Image Smoothing
TLDR
A deep neural network structure that exploits edge information in addressing representative low-level vision tasks such as layer separation and image filtering by estimating edges and reconstructing images using only cascaded convolutional layers arranged such that no handcrafted or application-specific image-processing components are required.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
TLDR
This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.
Benchmarking Single-Image Reflection Removal Algorithms
TLDR
This paper presents the first captured Single-image Reflection Removal dataset ‘SIR2’ with 40 controlled and 100 wild scenes, ground truth of background and reflection, and performs quantitative and visual quality comparisons for four state-of-the-art single-image reflection removal algorithms using four error metrics.
Photo-Realistic Single Image Super-Resolution Using a Generative Adversarial Network
TLDR
SRGAN, a generative adversarial network (GAN) for image super-resolution (SR), is presented, to its knowledge, the first framework capable of inferring photo-realistic natural images for 4x upscaling factors and a perceptual loss function which consists of an adversarial loss and a content loss.
Robust Separation of Reflection from Multiple Images
TLDR
A novel Augmented Lagrangian Multiplier based algorithm is designed to efficiently and effectively solve the decomposition problem and demonstrates the superior performance of the proposed method over the state of the arts, in terms of accuracy and simplicity.
Single Image Layer Separation Using Relative Smoothness
  • Yu Li, M. S. Brown
  • Computer Science
    2014 IEEE Conference on Computer Vision and Pattern Recognition
  • 2014
TLDR
This paper addresses extracting two layers from an image where one layer is smoother than the other by introducing a novel strategy that regularizes the gradients of the two layers such that one has a long tail distribution and the other a short tail distribution.
Photographic Image Synthesis with Cascaded Refinement Networks
  • Qifeng Chen, V. Koltun
  • Computer Science
    2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
TLDR
It is shown that photographic images can be synthesized from semantic layouts by a single feedforward network with appropriate structure, trained end-to-end with a direct regression objective.
Image-to-Image Translation with Conditional Adversarial Networks
TLDR
Conditional adversarial networks are investigated as a general-purpose solution to image-to-image translation problems and it is demonstrated that this approach is effective at synthesizing photos from label maps, reconstructing objects from edge maps, and colorizing images, among other tasks.
Fast Image Processing with Fully-Convolutional Networks
TLDR
This work presents an approach to accelerating a wide variety of image processing operators using a fully-convolutional network that is trained on input-output pairs that demonstrate the operator’s action, and demonstrates that the presented approach is significantly more accurate than prior approximation schemes.
Single Image Reflection Suppression
TLDR
This work proposes a novel approach to suppress reflections, based on a Laplacian data fidelity term and an l-zero gradient sparsity term imposed on the output, which performs better than the state-of-the-art reflection removal techniques.
...
...