Corpus ID: 214667483

TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation

@article{Jiang2020TLDRTL,
  title={TLDR: Token Loss Dynamic Reweighting for Reducing Repetitive Utterance Generation},
  author={Shaojie Jiang and Thomas Wolf and Christof Monz and M. Rijke},
  journal={ArXiv},
  year={2020},
  volume={abs/2003.11963}
}
Natural Language Generation (NLG) models are prone to generating repetitive utterances. In this work, we study the repetition problem for encoder-decoder models, using both recurrent neural network (RNN) and transformer architectures. To this end, we consider the chit-chat task, where the problem is more prominent than in other tasks that need encoder-decoder architectures. We first study the influence of model architectures. By using pre-attention and highway connections for RNNs, we manage to… Expand
5 Citations
Implicit Unlikelihood Training: Improving Neural Text Generation with Reinforcement Learning
  • Highly Influenced
  • PDF
Diverse Keyphrase Generation with Neural Unlikelihood Training
  • 1
  • PDF
Detecting and Classifying Malevolent Dialogue Responses: Taxonomy, Data and Methodology
  • 1
  • PDF
Feature-based detection of automated language models: tackling GPT-2, GPT-3 and Grover
  • Highly Influenced
  • PDF
Mirostat: A Perplexity-Controlled Neural Text Decoding Algorithm
  • 1
  • PDF

References

SHOWING 1-10 OF 42 REFERENCES
A Deep Reinforced Model for Abstractive Summarization
  • 797
  • PDF
Sequence to Sequence Learning with Neural Networks
  • 12,176
  • PDF
Neural Text Generation with Unlikelihood Training
  • 85
  • Highly Influential
  • PDF
Abstractive Text Summarization using Sequence-to-sequence RNNs and Beyond
  • 1,175
  • PDF
Attention is All you Need
  • 18,162
  • Highly Influential
  • PDF
Deep Reinforcement Learning for Dialogue Generation
  • 835
  • Highly Influential
  • PDF
CTRL: A Conditional Transformer Language Model for Controllable Generation
  • 248
  • Highly Influential
  • PDF
A Diversity-Promoting Objective Function for Neural Conversation Models
  • 1,017
  • Highly Influential
  • PDF
Effective Approaches to Attention-based Neural Machine Translation
  • 4,765
  • PDF
Modeling Coverage for Neural Machine Translation
  • 531
  • PDF
...
1
2
3
4
5
...