Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints

@inproceedings{Wu2021TrainingAC,
  title={Training Adaptive Computation for Open-Domain Question Answering with Computational Constraints},
  author={Yuxiang Wu and Pasquale Minervini and Pontus Stenetorp and Sebastian Riedel},
  booktitle={ACL/IJCNLP},
  year={2021}
}
Adaptive Computation (AC) has been shown to be effective in improving the efficiency of Open-Domain Question Answering (ODQA) systems. However, current AC approaches require tuning of all model parameters, and training state-of-the-art ODQA models requires significant computational resources that may not be available for most researchers. We propose Adaptive Passage Encoder, an AC method that can be applied to an existing ODQA model and can be trained efficiently on a single GPU. It keeps the… Expand
1 Citations

Figures and Tables from this paper

Approaches and Applications of Inductive Programming
In this report the program and the outcomes of Dagstuhl Seminar 21192 “Approaches and Applications of Inductive Programming” is documented. The goal of inductive programming (IP) is to induceExpand

References

SHOWING 1-10 OF 29 REFERENCES
A Discrete Hard EM Approach for Weakly Supervised Question Answering
TLDR
This paper develops a hard EM learning scheme that computes gradients relative to the most likely solution at each update and significantly outperforms previous methods on six QA tasks, including absolute gains of 2–10%, and achieves the state-of-the-art on five of them. Expand
The Right Tool for the Job: Matching Model and Instance Complexities
TLDR
This work proposes a modification to contextual representation fine-tuning which allows for an early (and fast) “exit” from neural network calculations for simple instances, and late (and accurate) exit for hard instances during inference. Expand
Multi-passage BERT: A Globally Normalized BERT Model for Open-domain Question Answering
TLDR
A multi-passage BERT model is proposed to globally normalize answer scores across all passages of the same question, and this change enables the QA model to find better answers by utilizing more passages. Expand
Passage Re-ranking with BERT
TLDR
A simple re-implementation of BERT for query-based passage re-ranking on the TREC-CAR dataset and the top entry in the leaderboard of the MS MARCO passage retrieval task, outperforming the previous state of the art by 27% in MRR@10. Expand
Latent Retrieval for Weakly Supervised Open Domain Question Answering
TLDR
It is shown for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system, and outperforming BM25 by up to 19 points in exact match. Expand
Distilling Knowledge from Reader to Retriever for Question Answering
TLDR
This paper proposes a technique to learn retriever models for downstream tasks, inspired by knowledge distillation, and which does not require annotated pairs of query and documents. Expand
Simple and Effective Multi-Paragraph Reading Comprehension
We consider the problem of adapting neural paragraph-level question answering models to the case where entire documents are given as input. Our proposed solution trains models to produce wellExpand
Reading Wikipedia to Answer Open-Domain Questions
TLDR
This approach combines a search component based on bigram hashing and TF-IDF matching with a multi-layer recurrent neural network model trained to detect answers in Wikipedia paragraphs, indicating that both modules are highly competitive with respect to existing counterparts. Expand
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
TLDR
It is shown that, in comparison to other recently introduced large-scale datasets, TriviaQA has relatively complex, compositional questions, has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and requires more cross sentence reasoning to find answers. Expand
End-to-End Open-Domain Question Answering with BERTserini
TLDR
An end-to-end question answering system that integrates BERT with the open-source Anserini information retrieval toolkit is demonstrated, showing that fine-tuning pretrained Bert with SQuAD is sufficient to achieve high accuracy in identifying answer spans. Expand
...
1
2
3
...