• Corpus ID: 211146563

Distributional Sliced-Wasserstein and Applications to Generative Modeling

@article{Nguyen2020DistributionalSA,
  title={Distributional Sliced-Wasserstein and Applications to Generative Modeling},
  author={Khai Nguyen and Nhat Ho and Tung Pham and Hung Hai Bui},
  journal={ArXiv},
  year={2020},
  volume={abs/2002.07367}
}
Sliced-Wasserstein distance (SWD) and its variation, Max Sliced-Wasserstein distance (Max-SWD), have been widely used in the recent years due to their fast computation and scalability when the probability measures lie in very high dimension. However, these distances still have their weakness, SWD requires a lot of projection samples because it uses the uniform distribution to sample projecting directions, Max-SWD uses only one projection, causing it to lose a large amount of information. In… 
[PDF] Semantic Reader
60 Citations
Highly Influential Citations
7
Background Citations
35
Methods Citations
24
Results Citations
4

Figures and Tables from this paper

60 Citations

Augmented Sliced Wasserstein Distances

This work proposes a new family of distance metrics, called augmented sliced Wasserstein distances (ASWDs), constructed by first mapping samples to higher-dimensional hypersurfaces parameterized by neural networks, and provides the condition under which the ASWD is a valid metric and shows it can be obtained by an injective neural network architecture.

Minimax confidence intervals for the Sliced Wasserstein distance

Confidence intervals for the Sliced Wasserstein distance are constructed which have finite-sample validity under no assumptions or under mild moment assumptions and are adaptive in length to the regularity of the underlying distributions.

Projection Robust Wasserstein Distance and Riemannian Optimization

A first step into a computational theory of the PRW distance is provided and the links between optimal transport and Riemannian optimization are provided.

On Projection Robust Optimal Transport: Sample Complexity and Model Misspecification

The viewpoint of projection robust (PR) OT is adopted, which seeks to maximize the OT cost between two measures by choosing a $k$-dimensional subspace onto which they can be projected, and an asymptotic guarantee of two types of minimum PRW estimators and a central limit theorem for max-sliced Wasserstein estimator under model misspecification are formulated.

Tessellated Wasserstein Auto-Encoders

A novel non-adversarial framework called Tessellated Wasserstein Auto-encoders (TWAE) is developed to tessellate the support of the target distribution into a given number of regions by the centroidal VoronoiTessellation (CVT) technique and design batches of data according to the tessellingation instead of random shuffling for accurate computation of discrepancy.

Encoded Prior Sliced Wasserstein AutoEncoder for learning latent manifold representations

This work introduces an Encoded Prior Sliced Wasserstein AutoEncoder (EPSWAE), wherein an additional prior-encoder network learns an unconstrained prior to match the encoded data manifold, and applies it to 3D-spiral, MNIST, and CelebA datasets, showing that its latent representations and interpolations are comparable to the state of the art on equivalent architectures.

Point-set Distances for Learning Representations of 3D Point Clouds

Experiments show that the sliced Wasserstein distance allows the neural network to learn a more efficient representation compared to the Chamfer discrepancy, which is demonstrated on several tasks in 3D computer vision including training a point cloud autoencoder, generative modeling, transfer learning, and point cloud registration.

BoMb-OT: On Batch of Mini-batches Optimal Transport

The proposed Batch of Mini-batches Optimal Transport (BoMb-OT) is a novel mini-batching scheme for optimal transport that can be formulated as a well-defined distance on the space of probability measures and provides a better objective loss than m-OT for doing approximate Bayesian computation, estimating parameters of interest in parametric generative models, and learning non-parametricGenerative models with gradient flow.

Restoration based Generative Models

The interpretation of DDMs in terms of image restoration (IR) is established and a multi-scale training is proposed, which improves the performance compared to the diffusion process, by taking advantage of the flexibility of the forward process.

Shedding a PAC-Bayesian Light on Adaptive Sliced-Wasserstein Distances

The PAC-Bayesian theory and the central observation that SW actually hinges on a slice-distribution-dependent Gibbs risk are leveraged to bring new contributions to this line of research.
...

References

SHOWING 1-10 OF 53 REFERENCES

Generalized Sliced Wasserstein Distances

The generalized Radon transform is utilized to define a new family of distances for probability measures, which are called generalized sliced-Wasserstein (GSW) distances, and it is shown that, similar to the SW distance, the GSW distance can be extended to a maximum GSW (max- GSW) distance.

Augmented Sliced Wasserstein Distances

This work proposes a new family of distance metrics, called augmented sliced Wasserstein distances (ASWDs), constructed by first mapping samples to higher-dimensional hypersurfaces parameterized by neural networks, and provides the condition under which the ASWD is a valid metric and shows it can be obtained by an injective neural network architecture.

Sliced Wasserstein Generative Models

  • Jiqing WuZhiwu Huang L. Gool
  • Computer Science
    2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
  • 2019
This paper proposes to approximate SWDs with a small number of parameterized orthogonal projections in an end-to-end deep learning fashion and designs two types of differentiable SWD blocks to equip modern generative frameworks---Auto-Encoders and Generative Adversarial Networks.

Asymptotic Guarantees for Learning Generative Models with the Sliced-Wasserstein Distance

A central limit theorem is proved, which characterizes the asymptotic distribution of the estimators and establishes a convergence rate of $\sqrt{n}$, where $n$ denotes the number of observed data points.

Max-Sliced Wasserstein Distance and Its Use for GANs

This work demonstrates that the recently proposed sliced Wasserstein distance trains GANs on high-dimensional images up to a resolution of 256x256 easily and develops the max-sliced Wasserenstein distance, which enjoys compelling sample complexity while reducing projection complexity, albeit necessitating a max estimation.

Generative Modeling Using the Sliced Wasserstein Distance

This work considers an alternative formulation for generative modeling based on random projections which, in its simplest form, results in a single objective rather than a saddle-point formulation and finds its approach to be significantly more stable compared to even the improved Wasserstein GAN.

Strong equivalence between metrics of Wasserstein type

The sliced Wasserstein and more recently max-sliced Wasserstein metrics $\mW_p$ have attracted abundant attention in data sciences and machine learning due to its advantages to tackle the curse of…

Projection Robust Wasserstein Distance and Riemannian Optimization

A first step into a computational theory of the PRW distance is provided and the links between optimal transport and Riemannian optimization are provided.

Subspace Robust Wasserstein distances

This work proposes a "max-min" robust variant of the Wasserstein distance by considering the maximal possible distance that can be realized between two measures, assuming they can be projected orthogonally on a lower $k$-dimensional subspace.

Sliced-Wasserstein Autoencoder: An Embarrassingly Simple Generative Model

Sliced-Wasserstein Autoencoders (SWAE) are introduced, which are generative models that enable one to shape the distribution of the latent space into any samplable probability distribution without the need for training an adversarial network or defining a closed-form for the distribution.
...