Bilingual-GAN: A Step Towards Parallel Text Generation

@article{Rashid2019BilingualGANAS,
  title={Bilingual-GAN: A Step Towards Parallel Text Generation},
  author={Ahmad Rashid and Alan Do-Omri and Md. Akmal Haidar and Qun Liu and Mehdi Rezagholizadeh},
  journal={ArXiv},
  year={2019},
  volume={abs/1904.04742}
}
Latent space based GAN methods and attention based sequence to sequence models have achieved impressive results in text generation and unsupervised machine translation respectively. [...] Key Method First two denoising autoencoders are trained, with shared encoders and back-translation to enforce a shared latent state between the two languages. The decoder is shared for the two translation directions. Next, a GAN is trained to generate synthetic "code" mimicking the languages' shared latent space.Expand
From Unsupervised Machine Translation to Adversarial Text Generation
TLDR
A self-attention based bilingual adversarial text generator which can learn to generate text from the encoder representation of an unsupervised neural machine translation system and generates more fluent sentences compared to monolingual baselines while effectively using half the number of parameters. Expand
Latent Code and Text-based Generative Adversarial Networks for Soft-text Generation
TLDR
This work introduces a novel text-based approach called Soft-GAN to effectively exploit GAN setup for text generation, and demonstrates how autoencoders (AEs) can be used for providing a continuous representation of sentences, which will be referred to as soft-text. Expand
Reconstructed similarity for faster GANs-based word translation to mitigate hubness
TLDR
A novel Reconstructed Similarity retrieval for the neural machine word translation model to mitigate the hubness problem regardless of whether it is a hub and construct a tiny GAN topology which achieves competitive results in the shortest time among GAN-based models, which easily trade off between speed and accuracy. Expand
Regularizing Transformers With Deep Probabilistic Layers
TLDR
This work uses a Gaussian Mixture Variational Autoencoder (GMVAE) as a regularizer layer and proves its effectiveness not only in Transformers but also in the most relevant encoder-decoder based LM, seq2seq with and without attention. Expand
Italian Counter Narrative Generation to Fight Online Hate Speech
TLDR
Results show that even for underresourced languages, data augmentation strategies paired with large unsupervised LMs can hold promising results, and strategy abbinate a potenti modelli del linguaggio possono offrire risultati promettenti. Expand
Synthetic Data for Deep Learning
TLDR
The synthetic-to-real domain adaptation problem that inevitably arises in applications of synthetic data is discussed, including synthetic- to-real refinement with GAN-based models and domain adaptation at the feature/model level without explicit data transformations. Expand

References

SHOWING 1-10 OF 39 REFERENCES
Unsupervised Machine Translation Using Monolingual Corpora Only
TLDR
This work proposes a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space and effectively learns to translate without using any labeled data. Expand
UNSUPERVISED MACHINE TRANSLATION USING MONOLINGUAL CORPORA ONLY
  • Piotr
  • Computer Science
  • 2017
TLDR
This work proposes a model that takes sentences from monolingual corpora in two different languages and maps them into the same latent space and effectively learns to translate without using any labeled data. Expand
Improving Neural Machine Translation with Conditional Sequence Generative Adversarial Nets
TLDR
Experimental results show that the proposed model consistently outperforms the traditional RNNSearch and the newly emerged state-of-the-art Transformer on English-German and Chinese-English translation tasks. Expand
TextKD-GAN: Text Generation Using Knowledge Distillation and Generative Adversarial Networks
TLDR
This work introduces a method using knowledge distillation to effectively exploit GAN setup for text generation and demonstrates how autoencoders (AEs) can be used for providing a continuous representation of sentences, which is a smooth representation that assign non-zero probabilities to more than one word. Expand
Phrase-Based & Neural Unsupervised Machine Translation
TLDR
This work investigates how to learn to translate when having access to only large monolingual corpora in each language, and proposes two model variants, a neural and a phrase-based model, which are significantly better than methods from the literature, while being simpler and having fewer hyper-parameters. Expand
Improving Neural Machine Translation Models with Monolingual Data
TLDR
This work pairs monolingual training data with an automatic back-translation, and can treat it as additional parallel training data, and obtains substantial improvements on the WMT 15 task English German, and for the low-resourced IWSLT 14 task Turkish->English. Expand
Unsupervised Neural Machine Translation
TLDR
This work proposes a novel method to train an NMT system in a completely unsupervised manner, relying on nothing but monolingual corpora, and consists of a slightly modified attentional encoder-decoder model that can be trained on monolingUAL corpora alone using a combination of denoising and backtranslation. Expand
On Accurate Evaluation of GANs for Language Generation
TLDR
It is demonstrated that the previously used BLEU score is not sensitive to semantic deterioration of generated texts and proposed alternative metrics that better capture the quality and diversity of the generated samples are proposed. Expand
Word Translation Without Parallel Data
TLDR
It is shown that a bilingual dictionary can be built between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way. Expand
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly. Expand
...
1
2
3
4
...