More Robust Dense Retrieval with Contrastive Dual Learning

  title={More Robust Dense Retrieval with Contrastive Dual Learning},
  author={Yizhi Li and Zhenghao Liu and Chenyan Xiong and Zhiyuan Liu},
  journal={Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval},
  • Yizhi Li, Zhenghao Liu, +1 author Zhiyuan Liu
  • Published 2021
  • Computer Science
  • Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval
Dense retrieval conducts text retrieval in the embedding space and has shown many advantages compared to sparse retrieval. Existing dense retrievers optimize representations of queries and documents with contrastive training and map them to the embedding space. The embedding space is optimized by aligning the matched query-document pairs and pushing the negative documents away from the query. However, in such training paradigm, the queries are only optimized to align to the documents and are… Expand

Figures and Tables from this paper


Learning Robust Dense Retrieval Models from Incomplete Relevance Labels
This paper improves upon ANCE by proposing a robust negative sampling strategy for scenarios where the training data lacks complete relevance annotations, and uses a small validation set with complete relevance judgments to accurately estimate a negative sampling distribution for dense retrieval models. Expand
Few-Shot Conversational Dense Retrieval
The analyses reveal that the advantages of ConvDR come from its ability to capture informative context while ignoring the unrelated context in previous conversation rounds, which makes ConvDR more effective as conversations evolve while previous systems may get confused by the increased noise from previous turns. Expand
A Deep Relevance Matching Model for Ad-hoc Retrieval
A novel deep relevance matching model (DRMM) for ad-hoc retrieval that employs a joint deep architecture at the query term level for relevance matching and can significantly outperform some well-known retrieval models as well as state-of-the-art deep matching models. Expand
End-to-End Neural Ad-hoc Ranking with Kernel Pooling
K-NRM uses a translation matrix that models word-level similarities via word embeddings, a new kernel-pooling technique that uses kernels to extract multi-level soft match features, and a learning-to-rank layer that combines those features into the final ranking score. Expand
An Introduction to Neural Information Retrieval
The monograph provides a complete picture of neural information retrieval techniques that culminate in supervised neural learning to rank models including deep neural network architectures that are trained end-to-end for ranking tasks. Expand
Latent Retrieval for Weakly Supervised Open Domain Question Answering
It is shown for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system, and outperforming BM25 by up to 19 points in exact match. Expand
Context-Aware Document Term Weighting for Ad-Hoc Search
Experiments show that an index using HDCT weights significantly improved the retrieval accuracy compared to typical term-frequency and state-of-the-art embedding-based indexes. Expand
Deeper Text Understanding for IR with Contextual Neural Language Modeling
Experimental results demonstrate that the contextual text representations from BERT are more effective than traditional word embeddings in bringing large improvements on queries written in natural languages. Expand
Convolutional Neural Networks for Soft-Matching N-Grams in Ad-hoc Search
Conv-KNRM uses Convolutional Neural Networks to represent n-grams of various lengths and soft matches them in a unified embedding space and is utilized by the kernel pooling and learning-to-rank layers to generate the final ranking score. Expand
Pre-training via Paraphrasing
It is shown that fine-tuning gives strong performance on a range of discriminative and generative tasks in many languages, making MARGE the most generally applicable pre-training method to date. Expand