Allen Institute for AI, University of Washington
Author pages are created from data sourced from our academic publisher partnerships and public sources.
Share This Author
Unsupervised Commonsense Question Answering with Self-Talk
- Vered Shwartz, Peter West, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
- Computer ScienceEMNLP
- 11 April 2020
An unsupervised framework based on self-talk as a novel alternative to multiple-choice commonsense tasks, inspired by inquiry-based discovery learning, which improves performance on several benchmarks and competes with models that obtain knowledge from external KBs.
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
This paper proposes a novel approach to unsupervised sentence summarization by mapping the Information Bottleneck principle to a conditional language modelling objective: given a sentence, the approach seeks a compressed sentence that can best predict the next sentence.
PLink: Discovering and Exploiting Locality for Accelerated Distributed Training on the public Cloud
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning
This paper proposes DeLorean, a new unsupervised decoding algorithm that can flexibly incorporate both the past and future contexts using only off-the-shelf, left-to-right language models and no supervision.
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
- Ari Holtzman, Peter West, Vered Schwartz, Yejin Choi, Luke Zettlemoyer
- Computer ScienceEMNLP
- 16 April 2021
Domain Conditional Pointwise Mutual Information is introduced, an alternative scoring function that directly compensates for surface form competition by simply reweighing each option according to a term that is proportional to its a priori likelihood within the context of the specific zero-shot task.
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
- Ximing Lu, Peter West, Rowan Zellers, Ronan Le Bras, Chandra Bhagavatula, Yejin Choi
- Computer ScienceNAACL
- 24 October 2020
This work proposes NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models – supervised or not – to generate fluent text while satisfying complex lexical constraints, and suggests the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.
Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference
- Galen Cassebeer Weld, Peter West, M. Glenski, D. Arbour, Ryan A. Rossi, Tim Althoff
- Computer ScienceArXiv
- 21 September 2020
The need for empirical evaluation frameworks for causal inference in natural language is demonstrated by showing that existing, commonly used models regularly disagree with one another on real world tasks.
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
It is shown that careful prompt engineering and a separately trained critic model allow us to selectively distill high-quality causal commonsense from GPT-3, a general language model, and this leads to a new framework, Symbolic Knowledge Distillation.
Generated Knowledge Prompting for Commonsense Reasoning
This work proposes generating knowledge statements directly from a language model with a generic prompt format, then selecting the knowledge which maximizes prediction probability, and finds that a model’s predictions can improve when using its own generated knowledge.
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models
- Peter West, Ximing Lu, Ari Holtzman, Chandra Bhagavatula, Jena D. Hwang, Yejin Choi
- Computer ScienceACL/IJCNLP
- 16 October 2020
Comprehensive empirical results demonstrate that REFLECTIVE DECODING outperforms strong unsupervised baselines on both paraphrasing and abductive text infilling, significantly narrowing the gap between unsuper supervised and supervised methods.