Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention

Abstract

In this paper, we proposed a sentence encoding-based model for recognizing text en-tailment. In our approach, the encoding of sentence is a two-stage process. Firstly, average pooling was used over word-level bidi-rectional LSTM (biLSTM) to generate a first-stage sentence representation. Secondly, attention mechanism was employed to replace average pooling on the same sentence for better representations. Instead of using target sentence to attend words in source sentence, we utilized the sentence's first-stage representation to attend words appeared in itself, which is called " Inner-Attention " in our paper. Experiments conducted on Stanford Natural Language Inference (SNLI) Corpus has proved the effectiveness of " Inner-Attention " mechanism. With less number of parameters, our model outperformed the existing best sentence encoding-based approach by a large margin.

Extracted Key Phrases

5 Figures and Tables

Showing 1-10 of 10 references

Lecture 6.5-rmsprop. COURS- ERA: Neural networks for machine learning

  • Tieleman, Geoffrey Hinton2012 ] Tijmen Tieleman, Hinton
  • 2012

Sanja Fidler, and Raquel Urtasun. 2015. Orderembeddings of images and language

  • Vendrov

Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification

  • Yang