Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs

@article{Yu2021TowardRF,
  title={Toward Realistic Face Photo–Sketch Synthesis via Composition-Aided GANs},
  author={Jun Yu and Xingxin Xu and Fei Gao and Shengjie Shi and Meng Wang and Dacheng Tao and Qingming Huang},
  journal={IEEE Transactions on Cybernetics},
  year={2021},
  volume={51},
  pages={4350-4362}
}
Face photo–sketch synthesis aims at generating a facial sketch/photo conditioned on a given photo/sketch. It covers wide applications including digital entertainment and law enforcement. Precisely depicting face photos/sketches remains challenging due to the restrictions on structural realism and textural consistency. While existing methods achieve compelling results, they mostly yield blurred effects and great deformation over various facial components, leading to the unrealistic feeling of… 

Figures and Tables from this paper

Face Photo-Sketch Synthesis Based on Conditional Adversarial Networks
TLDR
This paper proposes conditional generation adversarial networks (CGAN) based on arts drawing steps constrain and shows that, Compared with previous methods, the results generate visually comfortable face sketches and express more natural textures and details.
A Sketch-Transformer Network for Face Photo-Sketch Synthesis
TLDR
A Sketch-Transformer network for face photo-sketch synthesis is proposed, which consists of three closely-related modules, including a multi-scale feature and position encoder for patch-level feature and positions embedding, a self-attention module for capturing long-range spatial dependency, and amulti-scale spatially-adaptive de-normalization decoder for image reconstruction.
Biphasic Face Photo-Sketch Synthesis via Semantic-Driven Generative Adversarial Network with Graph Representation Learning
TLDR
A novel Semantic-Driven Generative Adversarial Network is proposed to address the above issues, cooperating with the Graph Representation Learning and a novel biphasic training strategy is proposed which is dedicated to refine the generated results through Iterative Cycle Training.
Face sketch to photo translation using generative adversarial networks
TLDR
This paper uses a pre-trained face photo generating model to synthesize high-quality natural face photos and employs an optimization procedure to keep high-fidelity to the input sketch and compares the results with state-of-the-art models quantitatively and qualitatively.
Face Sketch Synthesis via Semantic-Driven Generative Adversarial Network
TLDR
A novel Semantic-Driven Generative Adversarial Network (SDGAN) which embeds global structure-level style injection and local class-level knowledge re-weighting and dedicates to balance the contributions of different semantic classes is proposed.
Feature Space Based Loss for Face Photo-Sketch Synthesis
TLDR
This paper proposes a feature map based loss termed content feature loss, which are utilized to supervise the generator in the network to make the synthesized sketch/photo have more perceptual quality and can obtain better results in comparison with existing state-of-the-art methods.
Quality Guided Sketch-to-Photo Image Synthesis
TLDR
This work proposes a novel approach that adopts a generative adversarial network that synthesizes a single sketch into multiple synthetic images with unique attributes like hair color, sex, etc and incorporates multiple attribute assignment to the generator without compromising the identity of the synthesised image.
Multi-Scale Feature Channel Attention Generative Adversarial Network for Face Sketch Synthesis
TLDR
A fast Generative Adversarial Network with fast Multi-scale feature channel Attention, namely MAGAN is proposed, which can achieve better performance in both visual evaluations and quantitative evaluations (in terms of feature similarity and learned perceptual image patch similarity), compared with the state-of-the-art methods.
...
...

References

SHOWING 1-10 OF 70 REFERENCES
High-Quality Facial Photo-Sketch Synthesis Using Multi-Adversarial Networks
TLDR
A novel synthesis framework called Photo-Sketch Synthesis using Multi-Adversarial Networks, (PS2-MAN) that iteratively generates low resolution to high resolution images in an adversarial way that leverages the pair information using CycleGAN framework.
Face Sketch Synthesis by Multidomain Adversarial Learning
TLDR
This paper presents a novel face sketch synthesis method by multidomain adversarial learning (termed MDAL), which overcomes the defects of blurs and deformations toward high-quality synthesis.
Face Synthesis from Visual Attributes via Sketch using Conditional VAEs and GANs
TLDR
The proposed Attribute2Sketch2Face framework, which is based on a combination of deep Conditional Variational Autoencoder (CVAE) and Generative Adversarial Networks (GANs), consists of three stages: synthesis of facial sketch from attributes using a CVAE architecture, enhancement of coarse sketches to produce sharper sketches using a GAN-based framework, and Synthesis of face from sketch using another GAn-based network.
End-to-End Photo-Sketch Generation via Fully Convolutional Representation Learning
TLDR
This paper proposes a novel approach for photo-sketch generation, aiming to automatically transform face photos into detail-preserving personal sketches, and develops a fully convolutional network to learn the end-to-end photo-Sketch mapping.
Transductive Face Sketch-Photo Synthesis
TLDR
A novel transductive face sketch-photo synthesis method that incorporates the given test samples into the learning process and optimizes the performance on these test samples and efficiently optimizes this probabilistic model by alternating optimization.
Back projection: An effective postprocessing method for GAN-based face sketch synthesis
Multiple Representations-Based Face Sketch–Photo Synthesis
TLDR
A novel multiple representations-based face sketch-photo-synthesis method that adaptively combines multiple representations to represent an image patch that combines multiple features from face images processed using multiple filters and deploys Markov networks to exploit the interacting relationships between the neighboring image patches.
Compositional Model-Based Sketch Generator in Facial Entertainment
TLDR
The proposed compositional method crucially reduces the high frequency loss and improves the synthesis performance in comparison to the state-of-the-art methods.
Face Photo-Sketch Synthesis and Recognition
TLDR
A novel face photo-sketch synthesis and recognition method using a multiscale Markov Random Fields (MRF) model that allows effective matching between the two in face sketch recognition.
Markov Random Neural Fields for Face Sketch Synthesis
TLDR
A novel face sketch synthesis based on the Markov random neural fields including two structures, which can preserve the common structure and capture the characteristic features of the test photo, and achieves better results in terms of both quantitative and qualitative experimental evaluations.
...
...