Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks

@article{Severyn2015LearningTR,
  title={Learning to Rank Short Text Pairs with Convolutional Deep Neural Networks},
  author={Aliaksei Severyn and Alessandro Moschitti},
  journal={Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval},
  year={2015}
}
  • Aliaksei Severyn, Alessandro Moschitti
  • Published 9 August 2015
  • Computer Science
  • Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval
Learning a similarity function between pairs of objects is at the core of learning to rank approaches. In information retrieval tasks we typically deal with query-document pairs, in question answering -- question-answer pairs. However, before learning can take place, such pairs needs to be mapped from the original space of symbolic words into some feature space encoding various aspects of their relatedness, e.g. lexical, syntactic and semantic. Feature engineering is often a laborious task and… 

Figures and Tables from this paper

LSTM-based Deep Learning Models for Answer Ranking
TLDR
This paper defines a composite representation for questions and answers by combining convolutional neural network (CNN) with bidirectional long short-term memory (biLSTM) models, and learns a similarity function to relate them in a supervised way from the available training data.
Neural Text Embeddings for Information Retrieval
TLDR
This tutorial will introduce state-of-the-art neural embedding models and bridge the gap between these neural models with early representation learning approaches in IR (e.g., LSA), discuss some of the key challenges and insights in making these models work in practice, and demonstrate one of the toolsets available to researchers interested in this area.
Learning to Focus when Ranking Answers
TLDR
A novel ranking algorithm for question answering is proposed, QARAT, which uses an attention mechanism to learn on which words and phrases to focus when building the mutual representation of the query-answer pair.
Deep Learning for Information Retrieval
TLDR
This tutorial aims at summarizing and introducing the results of recent research on deep learning for information retrieval, in order to stimulate and foster more significant research and development work on the topic in the future.
Attention-based neural network for short-text question answering
TLDR
An attention-based convolutional neural network model is proposed to obtain word- and phrase-level interactive information and generate correct probability to re-rank candidate answers and results show that the proposed model can obtain a significant performance improvement.
Neural Models for Information Retrieval
TLDR
This tutorial introduces basic concepts and intuitions behind neural IR models, and places them in the context of traditional retrieval models, by introducing fundamental concepts of IR and different neural and non-neural approaches to learning vector representations of text.
Neural models for information retrieval without labeled data
TLDR
This dissertation addresses the gap between retrieval and recommendation models, as the two key components in most information systems, by introducing task-specific neural network architectures for a set of IR tasks and proposing novel unsupervised or weakly supervised solutions for training the models.
Learning to Rank with Query-level Semi-supervised Autoencoders
TLDR
This work first train a linear ranking model on the training data, and then incorporate the learned weights into the reconstruction costs of an autoencoder, and shows that the model can generate effective document features to improve the retrieval performance.
Neural Attention for Learning to Rank Questions in Community Question Answering
TLDR
This paper applies Long Short-Term Memory networks with an attention mechanism, which can select important parts of text for the task of similar question retrieval from community Question Answering (cQA) forums, and applies tree kernels to the filtered text representations, thus exploiting the implicit features of the subtree space for learning question reranking.
Simple Convolutional Neural Networks with Linguistically-Annotated Input for Answer Selection in Question Answering
TLDR
This research modifications the input that is fed to neural networks by annotating the input with linguistic information: POS tags, Named Entity Recognition output, linguistic relations, etc, and argues that this strikes a better balance between feature vs. network engineering.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 47 REFERENCES
Deep Learning for Answer Sentence Selection
TLDR
This work proposes a novel approach to solving the answer sentence selection task via means of distributed representations, and learns to match questions with answers by considering their semantic encoding.
Open Question Answering with Weakly Supervised Embedding Models
TLDR
This paper empirically demonstrate that the model can capture meaningful signals from its noisy supervision leading to major improvements over paralex, the only existing method able to be trained on similar weakly labeled data.
A unified architecture for natural language processing: deep neural networks with multitask learning
We describe a single convolutional neural network architecture that, given a sentence, outputs a host of language processing predictions: part-of-speech tags, chunks, named entity tags, semantic
Learning to Rank Answers to Non-Factoid Questions from Web Collections
TLDR
This work shows that it is possible to exploit existing large collections of question–answer pairs to extract such features and train ranking models which combine them effectively, providing one of the most compelling evidence to date that complex linguistic features such as word senses and semantic roles can have a significant impact on large-scale information retrieval tasks.
Modelling, Visualising and Summarising Documents with a Single Convolutional Neural Network
TLDR
A model is introduced that is able to represent the meaning of documents by embedding them in a low dimensional vector space, while preserving distinctions of word and sentence order crucial for capturing nuanced semantics.
A Neural Network for Factoid Question Answering over Paragraphs
TLDR
This work introduces a recursive neural network model, qanta, that can reason over question text input by modeling textual compositionality and applies it to a dataset of questions from a trivia competition called quiz bowl.
A Convolutional Neural Network for Modelling Sentences
TLDR
A convolutional architecture dubbed the Dynamic Convolutional Neural Network (DCNN) is described that is adopted for the semantic modelling of sentences and induces a feature graph over the sentence that is capable of explicitly capturing short and long-range relations.
Learning to rank: from pairwise approach to listwise approach
TLDR
It is proposed that learning to rank should adopt the listwise approach in which lists of objects are used as 'instances' in learning, and introduces two probability models, respectively referred to as permutation probability and top k probability, to define a listwise loss function for learning.
A syntax-aware re-ranker for microblog retrieval
TLDR
The results show that the L2R approach improves on almost all the participating systems at TREC, only using their raw scores as a single feature, and can easily be combined with those of other existing systems to boost their accuracy.
Probabilistic Tree-Edit Models with Structured Latent Variables for Textual Entailment and Question Answering
TLDR
This work captures the alignment by using a novel probabilistic model that models tree-edit operations on dependency parse trees and treats alignments as structured latent variables, and offers a principled framework for incorporating complex linguistic features.
...
1
2
3
4
5
...