The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translation

@article{Richardson2020TheSE,
  title={The Surprising Effectiveness of Linear Unsupervised Image-to-Image Translation},
  author={Eitan Richardson and Yair Weiss},
  journal={2020 25th International Conference on Pattern Recognition (ICPR)},
  year={2020},
  pages={7855-7861}
}
  • Eitan RichardsonYair Weiss
  • Published 24 July 2020
  • Computer Science
  • 2020 25th International Conference on Pattern Recognition (ICPR)
Unsupervised image-to-image translation is an inherently ill-posed problem. Recent methods based on deep encoder-decoder architectures have shown impressive results, but we show that they only succeed due to a strong locality bias, and they fail to learn very simple nonlocal transformations (e.g. mapping upside down faces to upright faces). When the locality bias is removed, the methods are too powerful and may fail to learn simple local transformations. In this paper we introduce linear… 

Encoding in Style: a StyleGAN Encoder for Image-to-Image Translation

We present a generic image-to-image translation framework, pixel2style2pixel (pSp). Our pSp framework is based on a novel encoder network that directly generates a series of style vectors which are

DEFENDING AGAINST IMAGE CORRUPTIONS THROUGH ADVERSARIAL AUGMENTATIONS

Classifiers trained using the proposed AdversarialAugment method in conjunction with prior methods improve upon the state-of-the-art on common image corruption benchmarks conducted in expectation on CIFAR-10-C and also improve worst-case performance against `p-norm bounded perturbations on both CIFar-10 and IMAGENET.

Domain Transformer: Predicting Samples of Unseen, Future Domains

  • Johannes Schneider
  • Computer Science
    2022 International Joint Conference on Neural Networks (IJCNN)
  • 2022
This work seeks to predict unseen data (and their labels) allowing us to tackle challenges s a non-constant data distribution in a proactive manner rather than detecting and reacting to already existing changes that might already have led to errors.

Defending Against Image Corruptions Through Adversarial Augmentations

This work proposes AdversarialAugment, a technique which optimizes the parameters of image-to-image models to generate adversarially corrupted augmented images and improves worst-case performance against `p-norm bounded perturbations on both CIFAR-10 and IMAGENET.

Improved Modeling of 3D Shapes with Multi-view Depth Maps

The quantitative evaluations show that the simple encoder-decoder framework is superior to existing depth map methods for reconstructing and synthesizing 3D objects and is competitive with other representations, such as point clouds, voxel grids, and implicit functions.

Spectral Geometric Matrix Completion

This work interprets the Deep Matrix Factorization model through the lens of spectral geometry, which allows it to incorporate explicit regularization without breaking the DMF structure, thus enjoying the best of both worlds.

StyleTalker: One-shot Style-based Audio-driven Talking Head Video Generation

. We propose StyleTalker , a novel audio-driven talking head generation model that can synthesize a video of a talking person from a single reference image with accurately audio-synced lip shapes,

Conditional Frechet Inception Distance

This work develops conditional versions of the Wasserstein metric and its Gaussian case known as the Frechet Inception Distance, and provides a closed form solution to the conditional FID (CFID) metric.

References

SHOWING 1-10 OF 18 REFERENCES

Multimodal Unsupervised Image-to-Image Translation

A Multimodal Unsupervised Image-to-image Translation (MUNIT) framework that assumes that the image representation can be decomposed into a content code that is domain-invariant, and a style code that captures domain-specific properties.

Few-Shot Unsupervised Image-to-Image Translation

This model achieves this few-shot generation capability by coupling an adversarial training scheme with a novel network design, and verifies the effectiveness of the proposed framework through extensive experimental validation and comparisons to several baseline methods on benchmark datasets.

Unsupervised Image-to-Image Translation Networks

This work makes a shared-latent space assumption and proposes an unsupervised image-to-image translation framework based on Coupled GANs that achieves state-of-the-art performance on benchmark datasets.

U-GAT-IT: Unsupervised Generative Attentional Networks with Adaptive Layer-Instance Normalization for Image-to-Image Translation

A novel method for unsupervised image- to-image translation, which incorporates a new attention module and a new learnable normalization function in an end-to-end manner, which can translate both images requiring holistic changes and images requiring large shape changes.

Fine-Grained Visual Comparisons with Local Learning

  • Aron YuK. Grauman
  • Computer Science
    2014 IEEE Conference on Computer Vision and Pattern Recognition
  • 2014
This work proposes a local learning approach for fine-grained visual comparisons that outperforms state-of-the-art methods for relative attribute prediction and shows how to identify analogous pairs using learned metrics.

Non-Adversarial Unsupervised Word Translation

A novel method is presented that first aligns the second moment of the word distributions of the two languages and then iteratively refines the alignment, which achieves better performance than recent state-of-the-art deep adversarial approaches and is competitive with the supervised baseline.

Deep Learning Face Attributes in the Wild

A novel deep learning framework for attribute prediction in the wild that cascades two CNNs, LNet and ANet, which are fine-tuned jointly with attribute tags, but pre-trained differently.

Unsupervised Correlation Analysis

  • Yedid HoshenLior Wolf
  • Computer Science
    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
  • 2018
This paper introduces a new method Unsupervised Correlation Analysis (UCA), which requires no prior correspondences between the two domains, and presents results on well accepted CCA benchmarks, showing that performance far exceeds other unsupervised baselines, and approaches supervised performance in some cases.

Adversarial Latent Autoencoders

Autoencoder networks are unsupervised approaches aiming at combining generative and representational properties by learning simultaneously an encoder-generator map. Although studied extensively, the

A Style-Based Generator Architecture for Generative Adversarial Networks

An alternative generator architecture for generative adversarial networks is proposed, borrowing from style transfer literature, that improves the state-of-the-art in terms of traditional distribution quality metrics, leads to demonstrably better interpolation properties, and also better disentangles the latent factors of variation.