High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network

@article{Liang2021HighResolutionPI,
  title={High-Resolution Photorealistic Image Translation in Real-Time: A Laplacian Pyramid Translation Network},
  author={Jie Liang and Huiyu Zeng and Lei Zhang},
  journal={2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
  year={2021},
  pages={9387-9395}
}
  • Jie LiangHuiyu ZengLei Zhang
  • Published 19 May 2021
  • Computer Science
  • 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
Existing image-to-image translation (I2IT) methods are either constrained to low-resolution images or long inference time due to their heavy computational burden on the convolution of high-resolution feature maps. In this paper, we focus on speeding-up the high-resolution photorealistic I2IT tasks based on closed-form Laplacian pyramid decomposition and reconstruction. Specifically, we reveal that the attribute transformations, such as illumination and color manipulation, relate more to the low… 
[PDF] Semantic Reader
28 Citations
Highly Influential Citations
7
Background Citations
16
Methods Citations
13

Figures and Tables from this paper

28 Citations

Multi-Curve Translator for High-Resolution Photorealistic Image Translation

. The dominant image-to-image translation methods are based on fully convolutional networks, which extract and translate an image’s features and then reconstruct the image. However, they have…

Multi-Curve Translator for Real-Time High-Resolution Image-to-Image Translation

The Multi-Curve Translator (MCT) is presented, which not only predicts the translated pixels for the corresponding input pixels but also for their neighboring pixels, which can dramatically lower the computational cost when working with high-resolution images.

Single UHD Image Dehazing via Interpretable Pyramid Network

This work introduces the principle of infinite approximation of Taylor’s theorem with the Laplace pyramid pattern to build a model which is capable of handling 4K hazy images in real-time and proposes a Tucker reconstructionbased regularization term that acts on each branch network of the pyramid model.

Global-Local Stepwise Generative Network for Ultra High-Resolution Image Restoration

A novel model for ultra high-resolution image restoration, referred to as the Global-Local Stepwise Generative Network ( GLSGN), which employs a stepwise restoring strategy involving four restoring pathways: three local pathways and one global pathway.

Image-to-Image Translation: Methods and Applications

The key techniques of the existing I2I works are analyzed and clarified to clarify the main progress the community has made and point out remaining challenges in related fields.

ABPN: Adaptive Blend Pyramid Network for Real-Time Local Retouching of Ultra High-Resolution Photo

A novel adaptive blend pyramid network that outperforms the existing methods by a large margin on two local photo retouching tasks and exhibits excellent performance in terms of running speed, achieving real-time inference on 4K images with a single NVIDIA Tesla P100 GPU is proposed.

Controllable Image Enhancement

This work first disentangle photo retouching skills from high-quality images and builds an efficient enhancement system for each skill, which achieves state-of-the-art performances on the benchmark dataset for image quality and model efficiency.

Fast and High-Quality Image Denoising via Malleable Convolutions

This work presents Malle able Conv olution ( MalleConv), which performs spatial-varying processing with minimal computational overhead, and uses it to build an efficient denoising network the authors call MalleNet, which achieves high-quality results without very deep architectures.

Fast and High Quality Image Denoising via Malleable Convolution

This work presents Malle able Conv olution ( MalleConv), which per-forms spatial-varying processing with minimal computational overhead, and uses it to build an efficient denoising network the authors call MalleNet, which achieves high-quality results without very deep architectures.

Laplacian Pyramid-like Autoencoder

LPAE improves the original results by combining the decoder part of the autoencoder and the super-resolution network and shows that the performance of the connected classification networks has remained substantially high.

References

SHOWING 1-10 OF 35 REFERENCES

Deep Laplacian Pyramid Networks for Fast and Accurate Super-Resolution

This paper proposes the Laplacian Pyramid Super-Resolution Network (LapSRN) to progressively reconstruct the sub-band residuals of high-resolution images and generates multi-scale predictions in one feed-forward pass through the progressive reconstruction, thereby facilitates resource-aware applications.

Deep bilateral learning for real-time image enhancement

This work introduces a new neural network architecture inspired by bilateral grid processing and local affine color transforms that processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of theart approximation techniques on a large class of image operators.

Photorealistic Style Transfer via Wavelet Transforms

This work proposes a wavelet corrected transfer based on whitening and coloring transforms (WCT2) that allows features to preserve their structural information and statistical properties of VGG feature space during stylization and provides a stable video stylization without temporal constraints.

Perceptual Losses for Real-Time Style Transfer and Super-Resolution

This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.

Learning Linear Transformations for Fast Image and Video Style Transfer

This work presents an approach for universal style transfer that learns the transformation matrix in a data-driven fashion that is efficient yet flexible to transfer different levels of styles with the same auto-encoder network.

Harmonic Unpaired Image-to-image Translation

This paper develops HarmonicGAN to learn bi-directional translations between the source and the target domains, and turns CycleGAN from a failure to a success, halving the mean-squared error, and generating images that radiologists prefer over competing methods in 95% of cases.

High-Resolution Image Synthesis and Semantic Manipulation with Conditional GANs

A new method for synthesizing high-resolution photo-realistic images from semantic label maps using conditional generative adversarial networks (conditional GANs) is presented, which significantly outperforms existing methods, advancing both the quality and the resolution of deep image synthesis and editing.

Image-to-image translation for cross-domain disentanglement

This paper achieves better results for translation on challenging datasets as well as for cross-domain retrieval on realistic datasets and compares the model to the state-of-the-art in multi-modal image translation.

The Laplacian Pyramid as a Compact Image Code

A technique for image encoding in which local operators of many scales but identical shape serve as the basis functions, which tends to enhance salient image features and is well suited for many image analysis tasks as well as for image compression.

Multimodal Unsupervised Image-to-Image Translation

A Multimodal Unsupervised Image-to-image Translation (MUNIT) framework that assumes that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties.