S3Net: A Single Stream Structure for Depth Guided Image Relighting

@article{Yang2021S3NetAS,
  title={S3Net: A Single Stream Structure for Depth Guided Image Relighting},
  author={Hao Yang and Wei-Ting Chen and Sy-Yen Kuo},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
  year={2021},
  pages={276-283}
}
  • Hao Yang, Wei-Ting Chen, Sy-Yen Kuo
  • Published 3 May 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)
Depth guided any-to-any image relighting aims to generate a relit image from the original image and corresponding depth maps to match the illumination setting of the given guided image and its depth map. To the best of our knowledge, this task is a new challenge that has not been addressed in the previous literature. To address this issue, we propose a deep learning-based neural Single Stream Structure network called S3Net for depth guided image relighting. This network is an encoder-decoder… Expand

Figures and Tables from this paper

Multi-modal Bifurcated Network for Depth Guided Image Relighting
TLDR
A deep learningbased method called multi-modal bifurcated network (MBNet) for depth guided image relighting, where given an image and the corresponding depth maps, a new image with the given illuminant angle and color temperature is generated by this model. Expand
LAFFNet: A Lightweight Adaptive Feature Fusion Network for Underwater Image Enhancement
TLDR
This work proposes a lightweight adaptive feature fusion network (LAFFNet) that effectively improves high-level vision tasks like salience object detection and single image depth estimation and outperforms state-of-the-art algorithms by extensive experiments. Expand
NTIRE 2021 Depth Guided Image Relighting Challenge
TLDR
This paper reviews the NTIRE 2021 depth guided image relighting challenge and relies on the VIDIT dataset for each of the two challenge tracks, including depth information, to transform the illumination settings of an input image to match those of another guide image, similar to style transfer. Expand
Robust Unsupervised Multi-Object Tracking in Noisy Environments
TLDR
The proposed single-head attention model AttU-Net shows better unsupervised MOT tracking performance over variational inference-based state-of-the-art baselines and helps limit the negative impact of noise by learning visual representations at different segment scales. Expand

References

SHOWING 1-10 OF 38 REFERENCES
NTIRE 2021 Depth Guided Image Relighting Challenge
TLDR
This paper reviews the NTIRE 2021 depth guided image relighting challenge and relies on the VIDIT dataset for each of the two challenge tracks, including depth information, to transform the illumination settings of an input image to match those of another guide image, similar to style transfer. Expand
Res2Net: A New Multi-Scale Backbone Architecture
TLDR
This paper proposes a novel building block for CNNs, namely Res2Net, by constructing hierarchical residual-like connections within one single residual block that represents multi-scale features at a granular level and increases the range of receptive fields for each network layer. Expand
VIDIT: Virtual Image Dataset for Illumination Transfer
TLDR
This work presents a novel dataset, the Virtual Image Dataset for Illumination Transfer (VIDIT), in an effort to create a reference evaluation benchmark and to push forward the development of illumination manipulation methods. Expand
Y-net: Multiscale feature aggregation network with wavelet structure similarity loss function for single image dehazing
  • IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020. 3, 4, 5, 7
  • 2020
A wavelet tour of signal processing
Introduction to a Transient World. Fourier Kingdom. Discrete Revolution. Time Meets Frequency. Frames. Wavelet Zoom. Wavelet Bases. Wavelet Packet and Local Cosine Bases. An Approximation Tour.Expand
SA-AE for Any-to-Any Relighting
WDRN : A Wavelet Decomposed RelightNet for Image Relighting
TLDR
A wavelet decomposed RelightNet called WDRN is proposed which is a novel encoder-decoder network employing wavelet based decomposition followed by convolution layers under a muti-resolution framework and a novel loss function called gray loss that ensures efficient learning of gradient in illumination along different directions of the ground truth image giving rise to visually superior relit images. Expand
Decoupled Weight Decay Regularization
TLDR
This work proposes a simple modification to recover the original formulation of weight decay regularization by decoupling the weight decay from the optimization steps taken w.r.t. the loss function, and provides empirical evidence that this modification substantially improves Adam's generalization performance. Expand
The Unreasonable Effectiveness of Deep Features as a Perceptual Metric
TLDR
A new dataset of human perceptual similarity judgments is introduced and it is found that deep features outperform all previous metrics by large margins on this dataset, and suggests that perceptual similarity is an emergent property shared across deep visual representations. Expand
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers. Expand
...
1
2
3
4
...