Continual Learning of Generative Models with Limited Data: From Wasserstein-1 Barycenter to Adaptive Coalescence

@article{Dedeoglu2021ContinualLO,
  title={Continual Learning of Generative Models with Limited Data: From Wasserstein-1 Barycenter to Adaptive Coalescence},
  author={Mehmet Dedeoglu and Sen Lin and Zhaofeng Zhang and Junshan Zhang},
  journal={ArXiv},
  year={2021},
  volume={abs/2101.09225}
}
Learning generative models is challenging for a network edge node with limited data and computing power. Since tasks in similar environments share model similarity, it is plausible to leverage pre-trained generative models from the cloud or other edge nodes. Appealing to optimal transport theory tailored towards Wasserstein-1 generative adversarial networks (WGAN), this study aims to develop a framework which systematically optimizes continual learning of generative models using local data at… 
[PDF] Semantic Reader

References

SHOWING 1-10 OF 62 REFERENCES

Generative Adversarial Nets

We propose a new framework for estimating generative models via an adversarial process, in which we simultaneously train two models: a generative model G that captures the data distribution, and a…

Decentralized Learning of Generative Adversarial Networks from Non-iid Data

A new decentralized approach for learning GANs from non-iid data called Forgiver-First Update (F2U), which asks clients to train an individual discriminator with their own data and updates a generator to fool the most `forgiving' discriminators who deem generated samples as the most real.

Transferring GANs: generating images from limited data

The results show that using knowledge from pretrained networks can shorten the convergence time and can significantly improve the quality of the generated images, especially when the target data is limited and it is suggested that density may be more important than diversity.

KE-GAN: Knowledge Embedded Generative Adversarial Networks for Semi-Supervised Scene Parsing

KE-GAN, a novel Knowledge Embedded Generative Adversarial Networks, is proposed to tackle the challenging problem of scene parsing in a semi-supervised fashion and is capable of improving semantic consistencies and learning better representations for scene parsing, resulting in the state-of theart performance.

MD-GAN: Multi-Discriminator Generative Adversarial Networks for Distributed Datasets

This paper proposes a novel learning procedure for GANs so that they fit this distributed setup, and exhibits a reduction by a factor of two of the learning complexity on each worker node, while providing better or identical performances with the adaptation of federated learning.

Generalizing to Unseen Domains via Adversarial Data Augmentation

This work proposes an iterative procedure that augments the dataset with examples from a fictitious target domain that is "hard" under the current model, and shows that the method is an adaptive data augmentation method where the authors append adversarial examples at each iteration.

A Geometric View of Optimal Transportation and Generative Model

Model Fusion via Optimal Transport

This work presents a layer-wise model fusion algorithm for neural networks that utilizes optimal transport to (soft-) align neurons across the models before averaging their associated parameters, and shows that this can successfully yield "one-shot" knowledge transfer between neural networks trained on heterogeneous non-i.i.d. data.

Parallel Streaming Wasserstein Barycenters

This work presents a scalable, communication-efficient, parallel algorithm for computing the Wasserstein barycenter of arbitrary distributions, which can operate directly on continuous input distributions and is optimized for streaming data.

Learning to Remember: A Synaptic Plasticity Driven Framework for Continual Learning

DGM relies on conditional generative adversarial networks with learnable connection plasticity realized with neural masking, and a dynamic network expansion mechanism is proposed that ensures sufficient model capacity to accommodate for continually incoming tasks.
...