Annotation Artifacts in Natural Language Inference Data

@inproceedings{Gururangan2018AnnotationAI,
  title={Annotation Artifacts in Natural Language Inference Data},
  author={Suchin Gururangan and Swabha Swayamdipta and Omer Levy and Roy Schwartz and Samuel R. Bowman and Noah A. Smith},
  booktitle={NAACL-HLT},
  year={2018}
}
Large-scale datasets for natural language inference are created by presenting crowd workers with a sentence (premise), and asking them to generate three new sentences (hypotheses) that it entails, contradicts, or is logically neutral with respect to. We show that, in a significant portion of such data, this protocol leaves clues that make it possible to identify the label by looking only at the hypothesis, without observing the premise. Specifically, we show that a simple text categorization… CONTINUE READING

Figures, Tables, and Topics from this paper.

Citations

Publications citing this paper.
SHOWING 1-10 OF 111 CITATIONS

Abductive Commonsense Reasoning

Chandra Bhagavatula, Ronan Le Bras, +6 authors Yejin Choi
  • ArXiv
  • 2019
VIEW 5 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

DRr-Net: Dynamic Re-Read Network for Sentence Semantic Matching

  • AAAI
  • 2019
VIEW 6 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Universal Adversarial Triggers for Attacking and Analyzing NLP

VIEW 4 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

Universal Adversarial Triggers for NLP

  • ArXiv
  • 2019
VIEW 4 EXCERPTS
CITES METHODS & BACKGROUND
HIGHLY INFLUENCED

FILTER CITATIONS BY YEAR

2016
2019

CITATION STATISTICS

  • 23 Highly Influenced Citations

  • Averaged 37 Citations per year from 2017 through 2019

  • 30% Increase in citations per year in 2019 over 2018

References

Publications referenced by this paper.