Show Your Work: Improved Reporting of Experimental Results

@article{Dodge2019ShowYW,
  title={Show Your Work: Improved Reporting of Experimental Results},
  author={Jesse Dodge and Suchin Gururangan and Dallas Card and Roy Schwartz and Noah A. Smith},
  journal={ArXiv},
  year={2019},
  volume={abs/1909.03004}
}
Research in natural language processing proceeds, in part, by demonstrating that new models achieve superior performance (e.g., accuracy) on held-out test data, compared to previous results. In this paper, we demonstrate that test-set performance scores alone are insufficient for drawing accurate conclusions about which model performs best. We argue for reporting additional details, especially performance on validation data obtained during model development. We present a novel technique for… CONTINUE READING

Citations

Publications citing this paper.

Green AI

VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

References

Publications referenced by this paper.