Improving Style Transfer with Calibrated Metrics

  title={Improving Style Transfer with Calibrated Metrics},
  author={Mao-Chuang Yeh and Shuai Tang and Anand Bhattad and Chuhang Zou and David Alexander Forsyth},
  journal={2020 IEEE Winter Conference on Applications of Computer Vision (WACV)},
Style transfer produces a transferred image which is a rendering of a content image in the manner of a style image. We seek to understand how to improve style transfer.To do so requires quantitative evaluation procedures, but current evaluation is qualitative, mostly involving user studies. We describe a novel quantitative evaluation procedure. Our procedure relies on two statistics: the Effectiveness (E) statistic measures the extent that a given style has been transferred to the target, and… Expand
Less is More, Faithful Style Transfer without Content Loss
The dominant style transfer framework is based on separately defining ‘style loss’ and ‘content loss’, then finding an image that trades off between minimizing both. The challenge of operating inExpand
Unrestricted Adversarial Examples via Semantic Manipulation
This paper introduces "unrestricted" perturbations that manipulate semantically meaningful image-based visual descriptors -- color and texture -- in order to generate effective and photorealistic adversarial examples. Expand
Parkinson’s Disease EMG Data Augmentation and Simulation with DCGANs and Style Transfer
Two new data augmentation approaches based on Deep Convolutional Generative Adversarial Networks (DCGANs) and Style Transfer for augmenting Parkinson’s Disease electromyography (EMG) signals are proposed. Expand
Style Transfer applied to CT image downscaling: a study case from Brazilian Coquinas
The identification of micropore systems in carbonate rocks is an important task of image processing because of the high impact these systems cause on fluid flow. Currently, one of the main tools usedExpand
Demographic Influences on Contemporary Art with Unsupervised Style Embeddings
This work presents contempArt, a multi-modal dataset of exclusively contemporary artworks, a collection of paintings and drawings, a detailed graph network based on social connections on Instagram and additional socio-demographic information; all attached to 442 artists at the beginning of their career. Expand


Universal Style Transfer via Feature Transforms
The key ingredient of the method is a pair of feature transforms, whitening and coloring, that are embedded to an image reconstruction network that reflects a direct matching of feature covariance of the content image to a given style image. Expand
Fast Patch-based Style Transfer of Arbitrary Style
A simpler optimization objective based on local matching that combines the content structure and style textures in a single layer of the pretrained network is proposed that has desirable properties such as a simpler optimization landscape, intuitive parameter tuning, and consistent frame-by-frame performance on video. Expand
Deep Photo Style Transfer
This paper introduces a deep-learning approach to photographic style transfer that handles a large variety of image content while faithfully transferring the reference style and constrain the transformation from the input to the output to be locally affine in colorspace. Expand
Neural Style Transfer: A Review
A taxonomy of current algorithms in the field of NST is proposed and several evaluation methods are presented and compared to compare different NST algorithms both qualitatively and quantitatively. Expand
Characterizing and Improving Stability in Neural Style Transfer
It is shown that the trace of the Gram matrix representing style is inversely related to the stability of the method, and a recurrent convolutional network is presented which incorporates a temporal consistency loss and overcomes the instability of prior methods. Expand
Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer
A multimodal convolutional neural network is proposed that takes into consideration faithful representations of both color and luminance channels, and performs stylization hierarchically with multiple losses of increasing scales, and can perform style transfer in nearly real-time by performing much more sophisticated training offline. Expand
Arbitrary Style Transfer in Real-Time with Adaptive Instance Normalization
This paper presents a simple yet effective approach that for the first time enables arbitrary style transfer in real-time, comparable to the fastest existing approach, without the restriction to a pre-defined set of styles. Expand
Improving the Neural Algorithm of Artistic Style
This work investigates different avenues of improving the Neural Algorithm of Artistic Style by modifying the style representation in order for it to capture more information and impose a tighter constraint on the style transfer result. Expand
Exploring the Neural Algorithm of Artistic Style
The method of style transfer presented in the article "A Neural Algorithm of Artistic Style" is explored and two new style representations built on top of network's features are proposed and discussed to achieve local and potentially content-aware style transfer. Expand
Controlling Perceptual Factors in Neural Style Transfer
The existing Neural Style Transfer method is extended to introduce control over spatial location, colour information and across spatial scale, enabling the combination of style information from multiple sources to generate new, perceptually appealing styles from existing ones. Expand