Deep Image Compositing

  title={Deep Image Compositing},
  author={He Zhang and Jianming Zhang and Federico Perazzi and Zhe L. Lin and Vishal M. Patel},
  journal={2021 IEEE Winter Conference on Applications of Computer Vision (WACV)},
Image compositing is a task of combining regions from different images to compose a new image. A common use case is background replacement of portrait images. To obtain high quality composites, professionals typically manually perform multiple editing steps such as segmentation, matting and foreground color decontamination, which is very time consuming even with sophisticated photo editing tools. In this paper, we propose a new method which can automatically generate high-quality image… 

Figures and Tables from this paper

Making Images Real Again: A Comprehensive Survey on Deep Image Composition
In this survey, the datasets and methods for the above research directions are summarized and the limitations and potential directions to facilitate the future research for image composition are discussed.
Total relighting
We propose a novel system for portrait relighting and background replacement, which maintains high-frequency boundary details and accurately synthesizes the subject's appearance as lit by novel
GALA: Toward Geometry-and-Lighting-Aware Object Search for Compositing
GALA (Geometry-and-Lighting-Aware), a generic foreground object search method with discriminative modeling on geometry and lighting compatibility for open- world image compositing achieves state-of-the-art results on the CAIS dataset and generalizes well on large-scale open-world datasets, i.e. Pixabay and Open Images.
Shadow Generation for Composite Image in Real-world Scenes
This work contributes a real-world shadow generation dataset DESOBA by generating synthetic composite images based on paired real images and deshadowed images and proposes a novel shadow generation network SGRNet, which consists of a shadow mask prediction stage and a shadow filling stage.
Multi-encoder Network for Parameter Reduction of a Kernel-based Interpolation Architecture
This paper presents a method for parameter reduction for a popular flow-less kernel-based network (Adaptive Collaboration of Flows), which reduces the number of parameters of the network and even achieves better performance compared to the original method.
Mask Guided Matting via Progressive Refinement Network
A robust matting framework that takes a general coarse mask as guidance, which can generalize to unseen types of guidance masks such as trimap and low-quality alpha matte, making it suitable for various application pipelines.
Temporally Consistent Relighting for Portrait Videos
This work proposes the first method to perform temporally consistent video portrait relighting for still images and demonstrates that this method outperforms previous work in balancing accurate relighting and temporal consistency on a number of real-world portrait videos.


Multi-scale image harmonization
This work presents a framework that explicitly matches the visual appearance of images through a process the authors call image harmonization, before blending them, and shows how the proposed framework can be used to produce realistic composites with minimal user interaction in a number of different scenarios.
Deep Image Harmonization
This work proposes an end-to-end deep convolutional neural network for image harmonization, which can capture both the context and semantic information of the composite images during harmonization and introduces an efficient way to collect large-scale and high-quality training data that can facilitate the training process.
Deep Image Matting
A novel deep learning based algorithm that can tackle image matting problems when an image has similar foreground and background colors or complicated textures and evaluation results demonstrate the superiority of this algorithm over previous methods.
Perceptual Losses for Real-Time Style Transfer and Super-Resolution
This work considers image transformation problems, and proposes the use of perceptual loss functions for training feed-forward networks for image transformation tasks, and shows results on image style transfer, where aFeed-forward network is trained to solve the optimization problem proposed by Gatys et al. in real-time.
GP-GAN: Towards Realistic High-Resolution Image Blending
This paper proposes a framework called Gaussian-Poisson Generative Adversarial Network (GP-GAN), which is the first work that explores the capability of GANs in high-resolution image blending task and achieves the state-of-the-art performance on Transient Attributes dataset.
Disentangled Image Matting
This paper proposes AdaMatting, a new end-to-end matting framework that disentangles this problem into two sub-tasks: trimap adaptation and alpha estimation, which achieves the state-of-the-art performance on Adobe Composition-1k dataset both qualitatively and quantitatively.
Deep Automatic Portrait Matting
An automatic image matting method for portrait images that does not need user interaction is proposed and achieves comparable results with state-of-the-art methods that require specified foreground and background regions or pixels.
Image and Video Matting: A Survey
This survey provides a comprehensive review of existing image and video matting algorithms and systems, with an emphasis on the advanced techniques that have been recently proposed.
The SYNTHIA Dataset: A Large Collection of Synthetic Images for Semantic Segmentation of Urban Scenes
This paper generates a synthetic collection of diverse urban images, named SYNTHIA, with automatically generated class annotations, and conducts experiments with DCNNs that show how the inclusion of SYnTHIA in the training stage significantly improves performance on the semantic segmentation task.
A Closed-Form Solution to Natural Image Matting
A closed-form solution to natural image matting that allows us to find the globally optimal alpha matte by solving a sparse linear system of equations and predicts the properties of the solution by analyzing the eigenvectors of a sparse matrix, closely related to matrices used in spectral image segmentation algorithms.