Mixture Content Selection for Diverse Sequence Generation

@article{Cho2019MixtureCS,
  title={Mixture Content Selection for Diverse Sequence Generation},
  author={Jaemin Cho and Minjoon Seo and Hannaneh Hajishirzi},
  journal={ArXiv},
  year={2019},
  volume={abs/1909.01953}
}
Generating diverse sequences is important in many NLP applications such as question generation or summarization that exhibit semantically one-to-many relationships between source and the target sequences. We present a method to explicitly separate diversification from generation using a general plug-and-play module (called SELECTOR) that wraps around and guides an existing encoder-decoder model. The diversification stage uses a mixture of experts to sample different binary masks on the source… CONTINUE READING

Figures, Tables, Results, and Topics from this paper.

Key Quantitative Results

  • In question generation (SQuAD) and abstractive summarization (CNN-DM), our method demonstrates significant improvements in accuracy, diversity and training efficiency, including state-of-the-art top-1 accuracy in both datasets, 6% gain in top-5 accuracy, and 3.7 times faster training over a state of the art model.