Dynamic Evaluation of Neural Sequence Models

@inproceedings{Krause2018DynamicEO,
  title={Dynamic Evaluation of Neural Sequence Models},
  author={Ben Krause and Emmanuel Kahembwe and Iain Murray and Steve Renals},
  booktitle={ICML},
  year={2018}
}
We explore dynamic evaluation, where sequence models are adapted to the recent sequence history using gradient descent, assigning higher probabilities to re-occurring sequential patterns. We develop a dynamic evaluation approach that outperforms existing adaptation approaches in our comparisons. We apply dynamic evaluation to outperform all previous word-level perplexities on the Penn Treebank and WikiText-2 datasets (achieving 51.1 and 44.3 respectively) and all previous character-level cross… CONTINUE READING
Highly Influential
This paper has highly influenced 10 other papers. REVIEW HIGHLY INFLUENTIAL CITATIONS
Highly Cited
This paper has 40 citations. REVIEW CITATIONS

8 Figures & Tables

Topics

Statistics

020406020172018
Citations per Year

Citation Velocity: 25

Averaging 25 citations per year over the last 2 years.

Learn more about how we calculate this metric in our FAQ.