A large annotated corpus for learning natural language inference

@inproceedings{Bowman2015ALA,
  title={A large annotated corpus for learning natural language inference},
  author={Samuel R. Bowman and Gabor Angeli and Christopher Potts and Christopher D. Manning},
  booktitle={EMNLP},
  year={2015}
}
Understanding entailment and contradiction is fundamental to understanding natural language, and inference about entailment and contradiction is a valuable testing ground for the development of semantic representations. However, machine learning research in this area has been dramatically limited by the lack of large-scale resources. To address this, we introduce the Stanford Natural Language Inference corpus, a new, freely available collection of labeled sentence pairs, written by humans doing… 

Figures and Tables from this paper

A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference
TLDR
The Multi-Genre Natural Language Inference corpus is introduced, a dataset designed for use in the development and evaluation of machine learning models for sentence understanding and shows that it represents a substantially more difficult task than does the Stanford NLI corpus.
Learning Entailment-Based Sentence Embeddings from Natural Language Inference
TLDR
This work proposes a simple interaction layer based on predefined entailment and contradiction scores applied directly to the sentence embeddings, which achieves results on natural language inference competitive with MLP-based models and directly represents the information needed for textual entailment.
Explaining Simple Natural Language Inference
TLDR
The experiment reveals several problems in the annotation guidelines, and various challenges of the NLI task itself, and leads to recommendations for future annotation tasks, for NLI and possibly for other tasks.
Towards a Unified Natural Language Inference Framework to Evaluate Sentence Representations
TLDR
This work generates a large-scale NLI dataset by recasting 11 existing datasets from 7 different semantic tasks, and uses this dataset of approximately half a million context-hypothesis pairs to test how well sentence encoders capture distinct semantic phenomena that are necessary for general language understanding.
Generating Natural Language Inference Chains
TLDR
A new task is proposed that measures how well a model can generate an entailed sentence from a source sentence and takes entailment-pairs of the Stanford Natural Language Inference corpus and trains an LSTM with attention, and applies this model recursively to input-output pairs, thereby generating natural language inference chains.
Probing the Natural Language Inference Task with Automated Reasoning Tools
TLDR
This work tests how well a machine-oriented controlled natural language (Attempto Controlled English) can be used to parse NLI sentences, and how well automated theorem provers can reason over the resulting formulae.
Syntactic Knowledge for Natural Language Inference in Portuguese
TLDR
This work proposes a new model for NLI that achieves 0.72 F\(_1\) score on ASSIN, setting a new state of the art, and shows that word embeddings and syntactic knowledge are both important to achieve such results.
Open IE-Triples Inference– Corpora Development and DNN Architectures
  • Martin Vita, P. Škoda
  • Computer Science
    2020 15th Conference on Computer Science and Information Systems (FedCSIS)
  • 2020
TLDR
This work proposes a method of transforming a general NLI corpus to an annotated corpus for relation inference that utilizes existing NLI annotations and introduces a novel relation inference corpus obtained from a well known SNLI corpus.
Natural Language Inference with Attentive Neural Networks
TLDR
This paper implements and evaluates a number of state-of-the-art attention based sequence models and observes that models using a matching-aggregation framework perform extremely well, but repeatedly make mistakes evaluating sentence pairs with ambiguity and irrelevant detail.
On the Evaluation of Semantic Phenomena in Neural Machine Translation Using Natural Language Inference
TLDR
A process for investigating the extent to which sentence representations arising from neural machine translation (NMT) systems encode distinct semantic phenomena is proposed, which finds its encoder appears most suited to supporting inferences at the syntax-semantics interface.
...
...

References

SHOWING 1-10 OF 38 REFERENCES
Recognising Textual Entailment with Logical Inference
TLDR
This work incorporates model building, a technique borrowed from automated reasoning, and shows that it is a useful robust method to approximate entailment, and uses machine learning to combine these deep semantic analysis techniques with simple shallow word overlap.
An extended model of natural logic
TLDR
A model of natural language inference which identifies valid inferences by their lexical and syntactic features, without full semantic interpretation is proposed, extending past work in natural logic by incorporating both semantic exclusion and implicativity.
Design and realization of a modular architecture for textual entailment
TLDR
The novel EXCITEMENT architecture, which was developed to enable and encourage the consolidation of methods and resources in the textual entailment area, decomposes RTE into components with strongly typed interfaces, and is available as open source software under the GNU General Public License.
A SICK cure for the evaluation of compositional distributional semantic models
TLDR
This work aims to help the research community working on compositional distributional semantic models (CDSMs) by providing SICK (Sentences Involving Compositional Knowldedge), a large size English benchmark tailored for them.
Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank
TLDR
A Sentiment Treebank that includes fine grained sentiment labels for 215,154 phrases in the parse trees of 11,855 sentences and presents new challenges for sentiment compositionality, and introduces the Recursive Neural Tensor Network.
Recursive Neural Networks Can Learn Logical Semantics
TLDR
This work generates artificial data from a logical grammar and uses it to evaluate the models' ability to learn to handle basic relational reasoning, recursive structures, and quantification, suggesting that they can learn suitable representations for logical inference in natural language.
Towards AI-Complete Question Answering: A Set of Prerequisite Toy Tasks
TLDR
This work argues for the usefulness of a set of proxy tasks that evaluate reading comprehension via question answering, and classify these tasks into skill sets so that researchers can identify (and then rectify) the failings of their systems.
Memory Networks
TLDR
This work describes a new class of learning models called memory networks, which reason with inference components combined with a long-term memory component; they learn how to use these jointly.
SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment
TLDR
This paper presents the task on the evaluation of Compositional Distributional Semantics Models on full sentences organized for the first time within SemEval2014, and attracted 21 teams, most of which participated in both subtasks.
Finding Contradictions in Text
TLDR
It is demonstrated that a system for contradiction needs to make more fine-grained distinctions than the common systems for entailment, and is argued for the centrality of event coreference and therefore incorporate such a component based on topicality.
...
...