• Publications
  • Influence
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
We introduce a new language representation model called BERT, which stands for Bidirectional Encoder Representations from Transformers. Expand
Deep contextualized word representations
TLDR
We introduce a new type of deep contextualized word representation that models both (1) complex characteristics of word use (e.g., syntax and semantics), and (2) how these uses vary across linguistic contexts (i.e., to model polysemy). Expand
End-to-end Neural Coreference Resolution
TLDR
We introduce the first end-to-end coreference resolution model and show that it significantly outperforms all previous work without using a syntactic parser or hand-engineered mention detector. Expand
Natural Questions: A Benchmark for Question Answering Research
TLDR
We present the Natural Questions corpus, a question answering data set. Expand
Deep Semantic Role Labeling: What Works and What's Next
TLDR
We introduce a new deep learning model for semantic role labeling (SRL) that significantly improves the state of the art, along with detailed analyses to reveal its strengths and limitations. Expand
Higher-order Coreference Resolution with Coarse-to-fine Inference
TLDR
We introduce a fully differentiable approximation to higher-order inference that uses the antecedent distribution from a span-ranking architecture as an attention mechanism to iteratively refine span representations. Expand
Latent Retrieval for Weakly Supervised Open Domain Question Answering
TLDR
We show for the first time that it is possible to jointly learn the retriever and reader from question-answer string pairs and without any IR system. Expand
BoolQ: Exploring the Surprising Difficulty of Natural Yes/No Questions
TLDR
We study yes/no questions that are naturally occurring — meaning that they are generated in unprompted and unconstrained settings. Expand
REALM: Retrieval-Augmented Language Model Pre-Training
TLDR
We augment language model pre- training with a latent knowledge retriever, which allows the model to retrieve and attend over documents from a large corpus such as Wikipedia, used during pre-training, fine-tuning and inference. Expand
Jointly Predicting Predicates and Arguments in Neural Semantic Role Labeling
TLDR
We propose an end-to-end approach for jointly predicting all predicates, arguments spans, and the relations between them. Expand
...
1
2
3
4
5
...