Corpus ID: 10894094

Improved Training of Wasserstein GANs

@inproceedings{Gulrajani2017ImprovedTO,
  title={Improved Training of Wasserstein GANs},
  author={Ishaan Gulrajani and Faruk Ahmed and Mart{\'i}n Arjovsky and Vincent Dumoulin and Aaron C. Courville},
  booktitle={NIPS},
  year={2017}
}
Generative Adversarial Networks (GANs) are powerful generative models, but suffer from training instability. [...] Key Method We propose an alternative to clipping weights: penalize the norm of gradient of the critic with respect to its input. Our proposed method performs better than standard WGAN and enables stable training of a wide variety of GAN architectures with almost no hyperparameter tuning, including 101-layer ResNets and language models over discrete data. We also achieve high quality generations on…Expand
A Wasserstein GAN model with the total variational regularization
TLDR
This paper combines a Total Variational (TV) regularizing term into the WGAN formulation instead of weight clipping or gradient penalty, which implies that the Lipschitz constraint is enforced on the critic network. Expand
Face Super-Resolution Through Wasserstein GANs
TLDR
Compared with other training objectives on a variety of GAN architectures in the context of single image super-resolution, the results agree that Wasserstein GAN with gradient penalty (WGAN-GP) provides stable and converging GAN training and thatWasserstein distance is an effective metric to gauge training progress. Expand
WASSERSTEIN-BOUNDED GENERATIVE ADVERSAR-
  • 2019
In the field of Generative Adversarial Networks (GANs), how to design a stable training strategy remains an open problem. Wasserstein GANs have largely promoted the stability over the original GANsExpand
Stabilizing Training of Generative Adversarial Networks through Regularization
TLDR
This work proposes a new regularization approach with low computational cost that yields a stable GAN training procedure and demonstrates the effectiveness of this regularizer accross several architectures trained on common benchmark image generation tasks. Expand
Improved Procedures for Training Primal Wasserstein GANs
  • T. Zhang, Zhongnian Li, Qi Zhu, Daoqiang Zhang
  • Computer Science
  • 2019 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computing, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI)
  • 2019
TLDR
The proposed procedures for improving the training of Primal Wasserstein GANs are tested on MNIST, CIFAR-10, LSUN-Bedroom and ImageNet-Dog category datasets, and it is demonstrated that the method is more time efficient compared with other generative model techniques. Expand
Language Modeling with Generative AdversarialNetworks
TLDR
This study presents the results of some experiments that indicate better training and convergence of Wasserstein GANs (WGANs) when a weaker regularization term is enforcing the Lipschitz constraint. Expand
Wasserstein GAN Can Perform PCA
  • Jaewoong Cho, Changho Suh
  • Computer Science, Mathematics
  • 2019 57th Annual Allerton Conference on Communication, Control, and Computing (Allerton)
  • 2019
TLDR
It is shown that Wasserstein GAN can approach the PCA solution in the limit of sample size, and hence it may serve as a basis for an optimal GAN architecture that yields the optimal generator for a wide range of data settings. Expand
Lp-WGAN: Using Lp-norm normalization to stabilize Wasserstein generative adversarial networks
TLDR
A novel method called Lp-WGAN is presented, where lp-norm normalization is employed to impose the constraints on the critic, and this method restricts the searching space of weights within a low-dimensional manifold and focuses on searching orientations of weights. Expand
Relaxed Wasserstein with Applications to GANs
TLDR
Experiments on real images demonstrate that the RWGAN with Kullback-Leibler (KL) cost function outperforms other competing approaches, e.g., WGANs, even with gradient penalty. Expand
Consistency Regularization for Generative Adversarial Networks
TLDR
This work proposes a simple, effective training stabilizer based on the notion of consistency regularization, which improves state-of-the-art FID scores for conditional generation and achieves the best F ID scores for unconditional image generation compared to other regularization methods on CIFAR-10 and CelebA. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 41 REFERENCES
Boundary-Seeking Generative Adversarial Networks
TLDR
This work introduces a method for training GANs with discrete data that uses the estimated difference measure from the discriminator to compute importance weights for generated samples, thus providing a policy gradient for training the generator. Expand
Least Squares Generative Adversarial Networks
TLDR
This paper proposes the Least Squares Generative Adversarial Networks (LSGANs) which adopt the least squares loss function for the discriminator, and shows that minimizing the objective function of LSGAN yields minimizing the Pearson X2 divergence. Expand
Maximum-Likelihood Augmented Discrete Generative Adversarial Networks
TLDR
This work derives a novel and low-variance GAN objective using the discriminator's output that follows corresponds to the log-likelihood, which is proved to be consistent in theory and beneficial in practice. Expand
Unrolled Generative Adversarial Networks
TLDR
This work introduces a method to stabilize Generative Adversarial Networks by defining the generator objective with respect to an unrolled optimization of the discriminator, and shows how this technique solves the common problem of mode collapse, stabilizes training of GANs with complex recurrent generators, and increases diversity and coverage of the data distribution by the generator. Expand
BEGAN: Boundary Equilibrium Generative Adversarial Networks
TLDR
This work proposes a new equilibrium enforcing method paired with a loss derived from the Wasserstein distance for training auto-encoder based Generative Adversarial Networks, which provides a new approximate convergence measure, fast and stable training and high visual quality. Expand
Improved Techniques for Training GANs
TLDR
This work focuses on two applications of GANs: semi-supervised learning, and the generation of images that humans find visually realistic, and presents ImageNet samples with unprecedented resolution and shows that the methods enable the model to learn recognizable features of ImageNet classes. Expand
f-GAN: Training Generative Neural Samplers using Variational Divergence Minimization
TLDR
It is shown that any f-divergence can be used for training generative neural samplers and the benefits of various choices of divergence functions on training complexity and the quality of the obtained generative models are discussed. Expand
Stacked Generative Adversarial Networks
TLDR
A novel generative model named Stacked Generative Adversarial Networks (SGAN), which is trained to invert the hierarchical representations of a bottom-up discriminative network, which is able to generate images of much higher quality than GANs without stacking. Expand
Improving Generative Adversarial Networks with Denoising Feature Matching
We propose an augmented training procedure for generative adversarial networks designed to address shortcomings of the original by directing the generator towards probable configurations of abstractExpand
Calibrating Energy-based Generative Adversarial Networks
TLDR
A flexible adversarial training framework is proposed, and it is proved this framework not only ensures the generator converges to the true data distribution, but also enables the discriminator to retain the density information at the global optimal. Expand
...
1
2
3
4
5
...