• Corpus ID: 2135897

Reasoning about Entailment with Neural Attention

@article{Rocktschel2016ReasoningAE,
  title={Reasoning about Entailment with Neural Attention},
  author={Tim Rockt{\"a}schel and Edward Grefenstette and Karl Moritz Hermann and Tom{\'a}s Kocisk{\'y} and Phil Blunsom},
  journal={CoRR},
  year={2016},
  volume={abs/1509.06664}
}
While most approaches to automatically recognizing entailment relations have used classifiers employing hand engineered features derived from complex natural language processing pipelines, in practice their performance has been only slightly better than bag-of-word pair classifiers using only lexical similarity. The only attempt so far to build an end-to-end differentiable neural network for entailment failed to outperform such a simple similarity classifier. In this paper, we propose a neural… 

Figures and Tables from this paper

Analysis of Deep Neural Networks for Textual Entailment Recognition
TLDR
This paper tries six architectures and analyzes their performance in terms of the performance parameters, and examines the impact of various layers like LSTM, concatenation, cosine similarity function, normalization, dropouts and attention.
A Sentence-to-Sentence Relation Network for Recognizing Textual Entailment
TLDR
A sentence encoding model that exploits the sentence-to-sentence relation information for RTE and combines the strength of RNN and CNN to present a unified model for the RTE task.
Joint Learning of Sentence Embeddings for Relevance and Entailment
TLDR
This work proposes a basic model to integrate evidence for entailment, shows that joint training of the sentence embeddings to model relevance and entailment is feasible even with no explicit per-evidence supervision, and shows the importance of evaluating strong baselines.
Embedding WordNet Knowledge for Textual Entailment
TLDR
This paper embeds the lexical entailment knowledge contained in WordNet in specially-learned word vectors, which they are called “entailment vectors,” and incorporates these entailment vectors into a decomposable attention model for textual entailment.
Recognizing Textual Entailment with Attentive Reading and Writing Operations
TLDR
This paper proposes to facilitate the conventional attentive reading operations with two sophisticated writing operations - forget and update - and writes the past attention information directly into the sentence representations so that higher memory capacity of attention history could be achieved.
Recognizing Textual Entailment via Multi-task Knowledge Assisted LSTM
TLDR
A deep neural network architecture called Multi-task Knowledge Assisted LSTM (MKAL) is proposed, which aims to conduct implicit inference with the assistant of KB and use predicate-to-predicate attention to detect the entailment between predicates.
Knowledge-aware Textual Entailment with Graph Attention Network
TLDR
A Knowledge-Context Interactive Textual Entailment Network (KCI-TEN) that learns graph level sentence representations by harnessing external knowledge graph with graph attention network and a text-graph interaction mechanism for neural based entailment matching learning that endows the redundancy and noise with less importance and put emphasis on the informative representations.
Visual Entailment: A Novel Task for Fine-Grained Image Understanding
TLDR
A new inference task, Visual Entailed (VE) - consisting of image-sentence pairs whereby a premise is defined by an image, rather than a natural language sentence as in traditional Textual Entailment tasks is introduced.
Generating Natural Language Inference Chains
TLDR
A new task is proposed that measures how well a model can generate an entailed sentence from a source sentence and takes entailment-pairs of the Stanford Natural Language Inference corpus and trains an LSTM with attention, and applies this model recursively to input-output pairs, thereby generating natural language inference chains.
AWE: Asymmetric Word Embedding for Textual Entailment
TLDR
Experimental results on SciTail and SNLI datasets show that the learned asymmetric word embeddings could significantly improve the word-word interaction based textual entailment models.
...
...

References

SHOWING 1-10 OF 36 REFERENCES
A large annotated corpus for learning natural language inference
TLDR
The Stanford Natural Language Inference corpus is introduced, a new, freely available collection of labeled sentence pairs, written by humans doing a novel grounded task based on image captioning, which allows a neural network-based model to perform competitively on natural language inference benchmarks for the first time.
Sequence to Sequence Learning with Neural Networks
TLDR
This paper presents a general end-to-end approach to sequence learning that makes minimal assumptions on the sequence structure, and finds that reversing the order of the words in all source sentences improved the LSTM's performance markedly, because doing so introduced many short term dependencies between the source and the target sentence which made the optimization problem easier.
Convolutional Neural Network Architectures for Matching Natural Language Sentences
TLDR
Convolutional neural network models for matching two sentences are proposed, by adapting the convolutional strategy in vision and speech and nicely represent the hierarchical structures of sentences with their layer-by-layer composition and pooling.
A Neural Attention Model for Abstractive Sentence Summarization
TLDR
This work proposes a fully data-driven approach to abstractive sentence summarization by utilizing a local attention-based model that generates each word of the summary conditioned on the input sentence.
SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment
TLDR
This paper presents the task on the evaluation of Compositional Distributional Semantics Models on full sentences organized for the first time within SemEval2014, and attracted 21 teams, most of which participated in both subtasks.
UNAL-NLP: Combining Soft Cardinality Features for Semantic Textual Similarity, Relatedness and Entailment
TLDR
These results confirm the results obtained in previous SemEval campaigns suggesting that the soft cardinality is a simple and useful tool for addressing a wide range of natural language processing problems.
End-To-End Memory Networks
TLDR
A neural network with a recurrent attention model over a possibly large external memory that is trained end-to-end, and hence requires significantly less supervision during training, making it more generally applicable in realistic settings.
Neural Machine Translation by Jointly Learning to Align and Translate
TLDR
It is conjecture that the use of a fixed-length vector is a bottleneck in improving the performance of this basic encoder-decoder architecture, and it is proposed to extend this by allowing a model to automatically (soft-)search for parts of a source sentence that are relevant to predicting a target word, without having to form these parts as a hard segment explicitly.
Grammar as a Foreign Language
TLDR
The domain agnostic attention-enhanced sequence-to-sequence model achieves state-of-the-art results on the most widely used syntactic constituency parsing dataset, when trained on a large synthetic corpus that was annotated using existing parsers.
ECNU: One Stone Two Birds: Ensemble of Heterogenous Measures for Semantic Relatedness and Textual Entailment
TLDR
This paper extracted seven types of features including text difference measures proposed in entailment judgement subtask, as well as common text similarity measures used in both subtasks to solve the both subtasking by considering them as a regression and a classification task respectively.
...
...