Corpus ID: 215238851

Evaluating the Evaluation of Diversity in Natural Language Generation

@inproceedings{Tevet2021EvaluatingTE,
  title={Evaluating the Evaluation of Diversity in Natural Language Generation},
  author={Guy Tevet and Jonathan Berant},
  booktitle={EACL},
  year={2021}
}
Despite growing interest in natural language generation (NLG) models that produce diverse outputs, there is currently no principled method for evaluating the diversity of an NLG system. In this work, we propose a framework for evaluating diversity metrics. The framework measures the correlation between a proposed diversity metric and a diversity parameter, a single parameter that controls some aspect of diversity in generated text. For example, a diversity parameter might be a binary variable… Expand
Decoding and Diversity in Machine Translation
Are Some Words Worth More than Others?
GenAug: Data Augmentation for Finetuning Text Generators
MultiTalk: A Highly-Branching Dialog Testbed for Diverse Conversations
4W1H Keyword Extraction based Summarization Model
Let Your Heart Speak in its Mother Tongue: Multilingual Captioning of Cardiac Signals
Neural Text Generation with Part-of-Speech Guided Softmax

References

SHOWING 1-10 OF 41 REFERENCES
Unifying Human and Statistical Evaluation for Natural Language Generation
Language GANs Falling Short
Evaluating the State-of-the-Art of End-to-End Natural Language Generation: The E2E NLG Challenge
Language Models are Unsupervised Multitask Learners
A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories
The Curious Case of Neural Text Degeneration
Diversity, Density, and Homogeneity: Quantitative Characteristic Metrics for Text Collections
ELI5: Long Form Question Answering
Evaluating Text GANs as Language Models
A Simple, Fast Diverse Decoding Algorithm for Neural Generation
...
1
2
3
4
5
...