Corpus ID: 52912377

Sinkhorn AutoEncoders

@article{Patrini2019SinkhornA,
  title={Sinkhorn AutoEncoders},
  author={Giorgio Patrini and Marcello Carioni and Patrick Forr'e and Samarth Bhargav and Max Welling and Rianne van den Berg and Tim Genewein and Frank Nielsen},
  journal={ArXiv},
  year={2019},
  volume={abs/1810.01118}
}
Optimal transport offers an alternative to maximum likelihood for learning generative autoencoding models. We show that minimizing the p-Wasserstein distance between the generator and the true data distribution is equivalent to the unconstrained min-min optimization of the p-Wasserstein distance between the encoder aggregated posterior and the prior in latent space, plus a reconstruction error. We also identify the role of its trade-off hyperparameter as the capacity of the generator: its… Expand
Generative Modeling with Optimal Transport Maps
TLDR
A minmax optimization algorithm is derived to efficiently compute OT maps for the quadratic cost (Wasserstein-2 distance) and this approach is extended to the case when the input and output distributions are located in the spaces of different dimensions and derive error bounds for the computed OT map. Expand
WiSE-ALE: Wide Sample Estimator for Approximate Latent Embedding
TLDR
This paper derives an alternative variational lower bound from the one common in VAEs, and demonstrates that WiSE-ALE can reach excellent reconstruction quality in comparison to other state-of-the-art VAE models, while still retaining the ability to learn a smooth, compact representation. Expand
Dual Rejection Sampling for Wasserstein Auto-Encoders
TLDR
A novel dual rejection sampling method is proposed to improve the performance of WAE on the generated samples in the sampling phase and corrects the generative prior by a discriminator based rejection sampling scheme in latent space and then rectifies the generated distribution by another discriminatorbased rejection sampling technique in data space. Expand
Momentum Contrastive Autoencoder: Using Contrastive Learning for Latent Space Distribution Matching in WAE
TLDR
This paper shows that using the contrastive learning framework to optimize the WAE loss achieves faster convergence and more stable optimization compared with existing popular algorithms for WAE. Expand
Learning from Nested Data with Ornstein Auto-Encoders
TLDR
The product-space OAE (PSOAE) is presented that minimizes a tighter upper bound of the distance and achieves orthogonality in the representation space and alleviates the instability of RIOAE and provides more flexible representation of nested data. Expand
AUTOENCODERS WITH SPHERICAL SLICED FUSED
Relational regularized autoencoder (RAE) is a framework to learn the distribution of data by minimizing a reconstruction loss together with a relational regularization on the latent space. A recentExpand
WiSE-VAE: Wide Sample Estimator VAE
TLDR
This paper derives an alternative variational lower bound from the one common in VAEs, and demonstrates that WiSE-VAE can reach excellent reconstruction quality in comparison to other state-of-the-art VAE models, while still retaining the ability to learn a smooth, compact representation. Expand
k-GANs: Ensemble of Generative Models with Semi-Discrete Optimal Transport
TLDR
A principled method for training an ensemble of GANs using semi-discrete optimal transport theory and the resulting k-GANs algorithm has strong theoretical connection with the k-medoids algorithm. Expand
Adversarial Networks and Autoencoders: The Primal-Dual Relationship and Generalization Bounds
TLDR
These findings present the first primal-dual relationship between GANs and Autoencoder models, comment on generalization abilities and make a step towards unifying these models. Expand
Improving Relational Regularized Autoencoders with Spherical Sliced Fused Gromov Wasserstein
TLDR
This work proposes a new relational discrepancy, named spherical sliced fused Gromov Wasserstein (SSFG), that can find an important area of projections characterized by a von Mises-Fisher distribution and introduces two variants of SSFG to improve its performance. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 70 REFERENCES
From optimal transport to generative modeling: the VEGAN cookbook
We study unsupervised generative modeling in terms of the optimal transport (OT) problem between true (but unknown) data distribution $P_X$ and the latent variable model distribution $P_G$. We showExpand
Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model
TLDR
Sliced-Wasserstein Autoencoders (SWAE) are introduced, which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution. Expand
Learning Generative Models with Sinkhorn Divergences
TLDR
This paper presents the first tractable computational method to train large scale generative models using an optimal transport loss, and tackles three issues by relying on two key ideas: entropic smoothing, which turns the original OT loss into one that can be computed using Sinkhorn fixed point iterations; and algorithmic (automatic) differentiation of these iterations. Expand
Wasserstein Auto-Encoders
TLDR
The Wasserstein Auto-Encoder (WAE) is proposed---a new algorithm for building a generative model of the data distribution that shares many of the properties of VAEs (stable training, encoder-decoder architecture, nice latent manifold structure) while generating samples of better quality, as measured by the FID score. Expand
Adversarial Autoencoders
TLDR
This paper shows how the adversarial autoencoder can be used in applications such as semi-supervised classification, disentangling style and content of images, unsupervised clustering, dimensionality reduction and data visualization, and performed experiments on MNIST, Street View House Numbers and Toronto Face datasets. Expand
GANs Trained by a Two Time-Scale Update Rule Converge to a Local Nash Equilibrium
TLDR
This work proposes a two time-scale update rule (TTUR) for training GANs with stochastic gradient descent on arbitrary GAN loss functions and introduces the "Frechet Inception Distance" (FID) which captures the similarity of generated images to real ones better than the Inception Score. Expand
VAE with a VampPrior
TLDR
This paper proposes to extend the variational auto-encoder (VAE) framework with a new type of prior called "Variational Mixture of Posteriors" prior, or VampPrior for short, which consists of a mixture distribution with components given by variational posteriors conditioned on learnable pseudo-inputs. Expand
Hyperspherical Variational Auto-Encoders
TLDR
This work proposes using a von Mises-Fisher distribution instead of a Gaussian distribution for both the prior and posterior of the Variational Auto-Encoder, leading to a hyperspherical latent space. Expand
Fixing a Broken ELBO
TLDR
This framework derives variational lower and upper bounds on the mutual information between the input and the latent variable, and uses these bounds to derive a rate-distortion curve that characterizes the tradeoff between compression and reconstruction accuracy. Expand
Distribution Matching in Variational Inference
TLDR
It is concluded that at present, VAE-GAN hybrids have limited applicability: they are harder to scale, evaluate, and use for inference compared to VAEs; and they do not improve over the generation quality of GANs. Expand
...
1
2
3
4
5
...