Annotation Artifacts in Natural Language Inference Data

@inproceedings{Gururangan2018AnnotationAI,
  title={Annotation Artifacts in Natural Language Inference Data},
  author={Suchin Gururangan and Swabha Swayamdipta and Omer Levy and Roy Schwartz and Samuel R. Bowman and Noah A. Smith},
  booktitle={NAACL-HLT},
  year={2018}
}
  • Suchin Gururangan, Swabha Swayamdipta, +3 authors Noah A. Smith
  • Published in NAACL-HLT 2018
  • Computer Science
  • Large-scale datasets for natural language inference are created by presenting crowd workers with a sentence (premise), and asking them to generate three new sentences (hypotheses) that it entails, contradicts, or is logically neutral with respect to. [...] Key Result Our findings suggest that the success of natural language inference models to date has been overestimated, and that the task remains a hard open problem.Expand Abstract

    Figures, Tables, and Topics from this paper.

    Uncertain Natural Language Inference
    1
    Explaining Simple Natural Language Inference
    1
    Generating Token-Level Explanations for Natural Language Inference
    8
    Stress Test Evaluation for Natural Language Inference
    77
    Don't Take the Premise for Granted: Mitigating Artifacts in Natural Language Inference
    15
    Learning Entailment-Based Sentence Embeddings from Natural Language Inference
    Mining Discourse Markers for Unsupervised Sentence Representation Learning
    10

    References

    Publications referenced by this paper.
    SHOWING 1-10 OF 33 REFERENCES
    A large annotated corpus for learning natural language inference
    1366
    Hypothesis Only Baselines in Natural Language Inference
    152
    Evaluating Compositionality in Sentence Embeddings
    53
    Natural Language Inference over Interaction Space
    134
    Discovery of inference rules for question-answering
    589
    SQuAD: 100, 000+ Questions for Machine Comprehension of Text
    2013
    A Decomposable Attention Model for Natural Language Inference
    691