• Corpus ID: 10894094

Improved Training of Wasserstein GANs

@inproceedings{Gulrajani2017ImprovedTO,
  title={Improved Training of Wasserstein GANs},
  author={Ishaan Gulrajani and Faruk Ahmed and Mart{\'i}n Arjovsky and Vincent Dumoulin and Aaron C. Courville},
  booktitle={NIPS},
  year={2017}
}
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. [] Key Method We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models over discrete data. We also achieve high quality generations on…
View PDF on arXiv
6,911 Citations
Highly Influential Citations
1,451
Background Citations
2,825
Methods Citations
3,550
Results Citations
131

Figures and Tables from this paper

6,911 Citations

A Wasserstein GAN model with the total variational regularization

This paper combines a Total Variational (TV) regularizing term into the WGAN formulation instead of weight clipping or gradient penalty, which implies that the Lipschitz constraint is enforced on the critic network.

Face Super-Resolution Through Wasserstein GANs

Compared with other training objectives on a variety of GAN architectures in the context of single image super-resolution, the results agree that Wasserstein GAN with gradient penalty (WGAN-GP) provides stable and converging GAN training and thatWasserstein distance is an effective metric to gauge training progress.

WASSERSTEIN-BOUNDED GENERATIVE ADVERSAR-

  • Computer Science
  • 2019
This paper presents a general framework named Wasserstein-Bounded GAN (WBGAN), which improves a large family of WGAN-based approaches by simply adding an upper-bound constraint to theWasserstein term.

Stabilizing Training of Generative Adversarial Networks through Regularization

This work proposes a new regularization approach with low computational cost that yields a stable GAN training procedure and demonstrates the effectiveness of this regularizer accross several architectures trained on common benchmark image generation tasks.

Improving GAN Training with Probability Ratio Clipping and Sample Reweighting

This work proposes a new variational GAN training framework which enjoys superior training stability and achieves significantly improved performance over a range of tasks, including text generation, text style transfer, and image generation.

Regularizing Generative Adversarial Networks under Limited Data

This work proposes a regularization approach for training robust GAN models on limited data and theoretically shows a connection between the regularized loss and an f-divergence called LeCam-Divergence, which is more robust under limited training data.

Language Modeling with Generative AdversarialNetworks

This study presents the results of some experiments that indicate better training and convergence of Wasserstein GANs (WGANs) when a weaker regularization term is enforcing the Lipschitz constraint.

Wasserstein GAN Can Perform PCA

  • Jaewoong ChoChangho Suh
  • Computer Science
    2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2019
It is shown that Wasserstein GAN can approach the PCA solution in the limit of sample size, and hence it may serve as a basis for an optimal GAN architecture that yields the optimal generator for a wide range of data settings.

Lp-WGAN: Using Lp-norm normalization to stabilize Wasserstein generative adversarial networks

Relaxed Wasserstein with Applications to GANs

Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty.
...

References

SHOWING 1-10 OF 37 REFERENCES

Boundary-Seeking Generative Adversarial Networks

This work introduces a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator.

Least Squares Generative Adversarial Networks

This paper proposes the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator, and shows that minimizing the objective function of LSGAN yields minimizing the Pearson X2 divergence.

Maximum-Likelihood Augmented Discrete Generative Adversarial Networks

This work derives a novel and low-variance GAN objective using the discriminator's output that follows corresponds to the log-likelihood, which is proved to be consistent in theory and beneficial in practice.

Unrolled Generative Adversarial Networks

This work introduces a method to stabilize Generative Adversarial Networks by defining the generator objective with respect to an unrolled optimization of the discriminator, and shows how this technique solves the common problem of mode collapse, stabilizes training of GANs with complex recurrent generators, and increases diversity and coverage of the data distribution by the generator.

BEGAN: Boundary Equilibrium Generative Adversarial Networks

This work proposes a new equilibrium enforcing method paired with a loss derived from the Wasserstein distance for training auto-encoder based Generative Adversarial Networks, which provides a new approximate convergence measure, fast and stable training and high visual quality.

Improved Techniques for Training GANs

This work focuses on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic, and presents ImageNet samples with unprecedented resolution and shows that the methods enable the model to learn recognizable features of ImageNet classes.

f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization

It is shown that any f-divergence can be used for training generative neural samplers and the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models are discussed.

Stacked Generative Adversarial Networks

A novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network, which is able to generate images of much higher quality than GANs without stacking.

Improving Generative Adversarial Networks with Denoising Feature Matching

We propose an augmented training procedure for generative adversarial networks designed to address shortcomings of the original by directing the generator towards probable configurations of abstract…

Generative Adversarial Nets

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a…