FaceShop: Deep Sketch-based Face Image Editing
@article{Portenier2018FaceShopDS, title={FaceShop: Deep Sketch-based Face Image Editing}, author={Tiziano Portenier and Qiyang Hu and Attila Szab{\'o} and Siavash Arjomand Bigdeli and Paolo Favaro and Matthias Zwicker}, journal={ArXiv}, year={2018}, volume={abs/1804.08972} }
We present a novel system for sketch-based face image editing, enabling users to edit images intuitively by sketching a few strokes on a region of interest. [] Key Method Our system is based on a novel sketch domain and a convolutional neural network trained end-to-end to automatically learn to render image regions corresponding to the input strokes. To achieve high quality and semantically consistent results we train our neural network on two simultaneous tasks, namely image completion and image translation…
Figures from this paper
121 Citations
Deep Generation of Face Images from Sketches
- Computer ScienceArXiv
- 2020
This work uses input sketches as soft constraints and is thus able to produce high-quality face images even from rough and/or incomplete sketches, and is easy to use even for non-artists, while still supporting fine-grained control of shape details.
SketchEdit: Mask-Free Local Image Manipulation with Partial Sketches
- Computer Science2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2022
This work investigates a new paradigm of sketch-based image manipulation: mask-free local image manipulation, which only requires sketch inputs from users and utilizes the entire original image.
DeepFacePencil: Creating Face Images from Freehand Sketches
- Computer ScienceACM Multimedia
- 2020
DeepFacePencil is proposed, an effective tool that is able to generate photo-realistic face images from hand-drawn sketches, based on a novel dual generator image translation network during training, designed to adaptively handle stroke distortions which are spatially varying to support various stroke styles and different level of details.
An Unpaired Sketch-to-Photo Translation Model
- Computer ScienceArXiv
- 2019
This work shows that the key to this task lies in decomposing the translation into two subtasks, shape translation and colorization, and proposes a model consisting of two sub-networks, with each one tackling one sub-task.
DeFLOCNet: Deep Image Editing via Flexible Low-level Controls
- Computer Science2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2021
This paper proposes DeFLOCNet which relies on a deep encoder-decoder CNN to retain the guidance of low-level controls in the deep feature representations of the CNN feature space and effectively transforms different user intentions to create visually pleasing content.
DeepFaceEditing: Deep Face Generation and Editing with Disentangled Geometry and Appearance Control
- Computer ScienceACM Trans. Graph.
- 2021
DeepFaceEditing, a structured disentanglement framework specifically designed for face images to support face generation and edit- ing with disentangled control of geometry and appearance, adopted a local-to-global approach to incorporate the face domain knowledge.
Fashion Editing with Multi-scale Attention Normalization
- Computer ScienceArXiv
- 2019
A novel Fashion Editing Generative Adversarial Network (FE-GAN), which is capable of manipulating fashion images by free-form sketches and sparse color strokes and significantly outperforms the state-of-the-art methods on image manipulation.
Unsupervised Sketch-to-Image Translation Network for Single Category Image Synthesis
- Computer Science2022 41st Chinese Control Conference (CCC)
- 2022
An unsupervised two-stage synthesis model to accomplish fine-grained sketch image translation at the pixel level is proposed, using the grayscale image as an intermediate result of the synthesis task between the geometric transformation and the color filling process.
Fashion Editing With Adversarial Parsing Learning
- Computer Science, Art2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
- 2020
A novel Fashion Editing Generative Adversarial Network (FE-GAN), which is capable of manipulating fashion images by free-form sketches and sparse color strokes and significantly outperforms the state-of-the-art methods on fashion image manipulation.
Deep Sketch-Guided Cartoon Video Inbetweening
- Computer ScienceIEEE Transactions on Visualization and Computer Graphics
- 2022
This work proposes a novel framework to produce cartoon videos by fetching the color information from two input keyframes while following the animated motion guided by a user sketch, and employing a blending module with occlusion estimation to synthesize the middle frame guided by the sketch.
References
SHOWING 1-10 OF 39 REFERENCES
Scribbler: Controlling Deep Image Synthesis with Sketch and Color
- Computer Science2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2017
A deep adversarial image synthesis architecture that is conditioned on sketched boundaries and sparse color strokes to generate realistic cars, bedrooms, or faces is proposed and demonstrates a sketch based image synthesis system which allows users to scribble over the sketch to indicate preferred color for objects.
Real-time user-guided image colorization with learned deep priors
- Computer ScienceACM Trans. Graph.
- 2017
We propose a deep learning approach for user-guided image colorization. The system directly maps a grayscale image, along with sparse, local user "hints" to an output colorization with a…
Automatic Photo Adjustment Using Deep Neural Networks
- Computer ScienceACM Trans. Graph.
- 2016
This article forms automatic photo adjustment in a manner suitable for deep neural networks, and introduces an image descriptor accounting for the local semantics of an image that can model local adjustments that depend on image semantics.
Painting style transfer for head portraits using convolutional neural networks
- Art, Computer ScienceACM Trans. Graph.
- 2016
This work presents a new technique for transferring the painting from a head portrait onto another and imposes novel spatial constraints by locally transferring the color distributions of the example painting to better captures the painting texture and maintains the integrity of facial structures.
A Neural Algorithm of Artistic Style
- ArtArXiv
- 2015
This work introduces an artificial system based on a Deep Neural Network that creates artistic images of high perceptual quality and offers a path forward to an algorithmic understanding of how humans create and perceive artistic imagery.
Fragment-based image completion
- MathematicsACM Trans. Graph.
- 2003
A new method for completing missing parts caused by the removal of foreground or background elements from an image, iteratively approximating the unknown regions and composites adaptive image fragments into the image to synthesize a complete, visually plausible and coherent image.
Deep bilateral learning for real-time image enhancement
- Computer ScienceACM Trans. Graph.
- 2017
This work introduces a new neural network architecture inspired by bilateral grid processing and local affine color transforms that processes high-resolution images on a smartphone in milliseconds, provides a real-time viewfinder at 1080p resolution, and matches the quality of state-of theart approximation techniques on a large class of image operators.
Let there be Color!: Joint End-to-end Learning of Global and Local Image Priors for Automatic Image Colorization with Simultaneous Classification
- Computer Science
- 2016
A novel technique to automatically colorize grayscale images that combines both global priors and local image features and can process images of any resolution, unlike most existing approaches based on CNN.
Photographic Image Synthesis with Cascaded Refinement Networks
- Computer Science2017 IEEE International Conference on Computer Vision (ICCV)
- 2017
It is shown that photographic images can be synthesized from semantic layouts by a single feedforward network with appropriate structure, trained end-to-end with a direct regression objective.
Region filling and object removal by exemplar-based image inpainting
- Computer ScienceIEEE Transactions on Image Processing
- 2004
The simultaneous propagation of texture and structure information is achieved by a single, efficient algorithm that combines the advantages of two approaches: exemplar-based texture synthesis and block-based sampling process.