• Publications
  • Influence
ERASER: A Benchmark to Evaluate Rationalized NLP Models
TLDR
This work proposes the Evaluating Rationales And Simple English Reasoning (ERASER) a benchmark to advance research on interpretable models in NLP, and proposes several metrics that aim to capture how well the rationales provided by models align with human rationales, and also how faithful these rationales are.
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
TLDR
This work collects human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation framework.
GeDi: Generative Discriminator Guided Sequence Generation
TLDR
GeDi is proposed as an efficient method for using smaller LMs as generative discriminators to guide generation from large LMs to make them safer and more controllable, and is found that GeDi gives stronger controllability than the state of the art method while also achieving generation speeds more than 30 times faster.
CoCo: Controllable Counterfactuals for Evaluating Dialogue State Trackers
TLDR
Human evaluations show that CoCo-generated conversations perfectly reflect the underlying user goal with more than 95% accuracy and are as human-like as the original conversations, further strengthening its reliability and promise to be adopted as part of the robustness evaluation of DST models.
Explaining and Improving Model Behavior with k Nearest Neighbor Representations
TLDR
This work proposes using k nearest neighbor (kNN) representations to identify training examples responsible for a model's predictions and obtains a corpus-level understanding of the model's behavior, and shows that the kNN approach makes the finetuned model more robust to adversarial inputs.
FastIF: Scalable Influence Functions for Efficient Model Interpretation and Debugging
TLDR
FASTIF, a set of simple modifications to influence functions that significantly improves their run-time, uses k-Nearest Neighbors to narrow the search space down to a subset of good candidate data points, identify the configurations that best balance the speed-quality trade-off in estimating the inverse Hessian-vector product, and introduces a fast parallel variant.
Robustness Gym: Unifying the NLP Evaluation Landscape
TLDR
Robustness Gym (RG), a simple and extensible evaluation toolkit that unifies 4 standard evaluation paradigms: subpopulations, transformations, evaluation sets, and adversarial attacks, is proposed.
Stacked Ensembles of Information Extractors for Knowledge-Base Population
TLDR
It is demonstrated that the stacking approach outperforms the best system from the2014 KBPESF competition as well as alternative ensembling methods employed in the 2014 KBP Slot Filler Validation task and several other ensembled baselines.
CTRLsum: Towards Generic Controllable Text Summarization
TLDR
CTRLsum enables users to control multiple aspects of generated summaries by interacting with the summarization system through textual input in the form of a set of keywords or descriptive prompts, and achieves state-of-the-art results on the CNN/DailyMail dataset.
DART: Open-Domain Structured Data Record to Text Generation
TLDR
The dataset construction framework effectively merged heterogeneous sources from open domain semantic parsing and spoken dialogue systems by utilizing techniques including tree ontology annotation, question-answer pair to declarative sentence conversion, and predicate unification, all with minimum post-editing.
...
1
2
3
4
5
...