• Publications
  • Influence
Don’t Stop Pretraining: Adapt Language Models to Domains and Tasks
TLDR
It is consistently found that multi-phase adaptive pretraining offers large gains in task performance, and it is shown that adapting to a task corpus augmented using simple data selection strategies is an effective alternative, especially when resources for domain-adaptive pretraining might be unavailable. Expand
Quoref: A Reading Comprehension Dataset with Questions Requiring Coreferential Reasoning
TLDR
This work presents a new crowdsourced dataset containing more than 24K span-selection questions that require resolving coreference among entities in over 4.7K English paragraphs from Wikipedia, and shows that state-of-the-art reading comprehension models perform significantly worse than humans on this benchmark. Expand
Explaining NLP Models via Minimal Contrastive Editing (MiCE)
TLDR
It is demonstrated how MICE edits can be used for two use cases in NLP system development—debugging incorrect model outputs and uncovering dataset artifacts—and thereby illustrate that producing contrastive explanations is a promising research direction for model interpretability. Expand
A Mention-Ranking Model for Abstract Anaphora Resolution
TLDR
A mention-ranking model that learns how abstract anaphors relate to their antecedents with an LSTM-Siamese Net is proposed and its model outperforms state-of-the-art results on shell noun resolution and reports first benchmark results on an abstractAnaphora subset of the ARRAU corpus. Expand
SRL4ORL: Improving Opinion Role Labeling Using Multi-Task Learning with Semantic Role Labeling
TLDR
It is found that the vanilla MTL model, which makes predictions using only shared ORL and SRL features, performs the best, and two MTL models improve significantly over the single-task model for labeling of both holders and targets, on the development and the test sets. Expand
Natural Language Rationales with Full-Stack Visual Reasoning: From Pixels to Semantic Frames to Commonsense Graphs
TLDR
This study presents RationaleˆVT Transformer, an integrated model that learns to generate free-text rationales by combining pretrained language models with object recognition, grounded visual semantic frames, and visual commonsense graphs, and finds that integration of richer semantic and pragmatic visual features improves visual fidelity of rationales. Expand
Teach Me to Explain: A Review of Datasets for Explainable NLP
TLDR
This review identifies three predominant classes of explanations (highlights, free-text, and structured), organize the literature on annotating each type, point to what has been learned to date, and give recommendations for collecting EXNLP datasets in the future. Expand
Measuring Association Between Labels and Free-Text Rationales
TLDR
It is demonstrated that existing models for faithful interpretability do not extend cleanly to tasks where free-text rationales are needed, and proposed measurements of label-rationale association, a necessary property of faithful rationales, are proposed for these models. Expand
Documenting the English Colossal Clean Crawled Corpus
TLDR
This work provides some of the first documentation of the English Colossal Clean Crawled Corpus (C4), one of the largest corpora of text available, and hosts an indexed version of C4 at https://c4-search.allenai.org/, allowing anyone to search it. Expand
Promoting Graph Awareness in Linearized Graph-to-Text Generation
TLDR
This work uses graph-denoising objectives implemented in a multi-task text-to-text framework and finds that these denoising scaffolds lead to substantial improvements in downstream generation in low-resource settings. Expand
...
1
2
...