JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video

  title={JSI-GAN: GAN-Based Joint Super-Resolution and Inverse Tone-Mapping with Pixel-Wise Task-Specific Filters for UHD HDR Video},
  author={Soo Ye Kim and Jihyong Oh and Munchurl Kim},
Joint learning of super-resolution (SR) and inverse tone-mapping (ITM) has been explored recently, to convert legacy low resolution (LR) standard dynamic range (SDR) videos to high resolution (HR) high dynamic range (HDR) videos for the growing need of UHD HDR TV/broadcasting applications. However, previous CNN-based methods directly reconstruct the HR HDR frames from LR SDR frames, and are only trained with a simple L2 loss. In this paper, we take a divide-and-conquer approach in designing a… 

Figures and Tables from this paper

Joint Super-Resolution and Inverse Tone-Mapping: A Feature Decomposition Aggregation Network and A New Benchmark
A lightweight Feature Decomposition Aggregation Network (FDAN) is designed, which can achieve learnable separation of feature details and contrasts and build up a Hierarchical FeatureDecomposition Group for powerful multi-level feature decomposition.
Self-Supervised Super-Resolution for Multi-Exposure Push-Frame Satellites
This work proposes a super-resolution method that can handle the signal-dependent noise in the inputs, process sequences of any length, and be robust to inaccuracies in the exposure times, and can be trained end-to-end with self-supervision, which makes it especially suited to handle real data.


Enhanced Deep Residual Networks for Single Image Super-Resolution
This paper develops an enhanced deep super-resolution network (EDSR) with performance exceeding those of current state-of-the-art SR methods, and proposes a new multi-scale deepsuper-resolution system (MDSR) and training method, which can reconstruct high-resolution images of different upscaling factors in a single model.
A Multi-purpose Convolutional Neural Network for Simultaneous Super-Resolution and High Dynamic Range Image Reconstruction
A convolutional neural network based structure for the joint learning of super-resolution and inverse tone-mapping is proposed, which can be used for converting LR-SDR legacy video to high resolution (HR) HDR video.
Rectifier Nonlinearities Improve Neural Network Acoustic Models
This work explores the use of deep rectifier networks as acoustic models for the 300 hour Switchboard conversational speech recognition task, and analyzes hidden layer representations to quantify differences in how ReL units encode inputs as compared to sigmoidal units.
Enhanced Pix2pix Dehazing Network
The proposed Enhanced Pix2pix Dehazing Network (EPDN), which generates a haze-free image without relying on the physical scattering model, is embedded by a generative adversarial network, which is followed by a well-designed enhancer.
Deep SR-ITM: Joint Learning of Super-Resolution and Inverse Tone-Mapping for 4K UHD HDR Applications
This paper proposes a joint super-resolution (SR) and inverse tone-mapping (ITM) framework, called Deep SR-ITM, which learns the direct mapping from LR SDR video to their HR HDR version, and shows good subjective quality with increased contrast and details, outperforming the previous joint SR and ITM method.
Deep Video Super-Resolution Network Using Dynamic Upsampling Filters Without Explicit Motion Compensation
A novel end-to-end deep neural network that generates dynamic upsampling filters and a residual image, which are computed depending on the local spatio-temporal neighborhood of each pixel to avoid explicit motion compensation is proposed.
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks
This work thoroughly study three key components of SRGAN – network architecture, adversarial loss and perceptual loss, and improves each of them to derive an Enhanced SRGAN (ESRGAN), which achieves consistently better visual quality with more realistic and natural textures than SRGAN.
Image Blind Denoising with Generative Adversarial Network Based Noise Modeling
A novel two-step framework is proposed, in which a Generative Adversarial Network is trained to estimate the noise distribution over the input noisy images and to generate noise samples to train a deep Convolutional Neural Network for denoising.
The relativistic discriminator: a key element missing from standard GAN
It is shown that RGANs and RaGANs are significantly more stable and generate higher quality data samples than their non-relativistic counterparts, and Standard RaGAN with gradient penalty generate data of better quality than WGAN-GP while only requiring a single discriminator update per generator update.
Deep Photo Enhancer: Unpaired Learning for Image Enhancement from Photographs with GANs
An unpaired learning method for image enhancement based on the framework of two-way generative adversarial networks (GANs) with several improvements that significantly improve the stability of GAN training for this application.