• Publications
  • Influence
Unsupervised Commonsense Question Answering with Self-Talk
TLDR
An unsupervised framework based on self-talk as a novel alternative to multiple-choice commonsense tasks, inspired by inquiry-based discovery learning, which improves performance on several benchmarks and competes with models that obtain knowledge from external KBs.
BottleSum: Unsupervised and Self-supervised Sentence Summarization using the Information Bottleneck Principle
TLDR
This paper proposes a novel approach to unsupervised sentence summarization by mapping the Information Bottleneck principle to a conditional language modelling objective: given a sentence, the approach seeks a compressed sentence that can best predict the next sentence.
Back to the Future: Unsupervised Backprop-based Decoding for Counterfactual and Abductive Commonsense Reasoning
TLDR
This paper proposes DeLorean, a new unsupervised decoding algorithm that can flexibly incorporate both the past and future contexts using only off-the-shelf, left-to-right language models and no supervision.
Surface Form Competition: Why the Highest Probability Answer Isn't Always Right
TLDR
Domain Conditional Pointwise Mutual Information is introduced, an alternative scoring function that directly compensates for surface form competition by simply reweighing each option according to a term that is proportional to its a priori likelihood within the context of the specific zero-shot task.
NeuroLogic Decoding: (Un)supervised Neural Text Generation with Predicate Logic Constraints
TLDR
This work proposes NeuroLogic Decoding, a simple yet effective algorithm that enables neural language models – supervised or not – to generate fluent text while satisfying complex lexical constraints, and suggests the limit of large-scale neural networks for fine-grained controllable generation and the promise of inference-time algorithms.
Adjusting for Confounders with Text: Challenges and an Empirical Evaluation Framework for Causal Inference
TLDR
The need for empirical evaluation frameworks for causal inference in natural language is demonstrated by showing that existing, commonly used models regularly disagree with one another on real world tasks.
Symbolic Knowledge Distillation: from General Language Models to Commonsense Models
TLDR
It is shown that careful prompt engineering and a separately trained critic model allow us to selectively distill high-quality causal commonsense from GPT-3, a general language model, and this leads to a new framework, Symbolic Knowledge Distillation.
Generated Knowledge Prompting for Commonsense Reasoning
TLDR
This work proposes generating knowledge statements directly from a language model with a generic prompt format, then selecting the knowledge which maximizes prediction probability, and finds that a model’s predictions can improve when using its own generated knowledge.
Reflective Decoding: Beyond Unidirectional Generation with Off-the-Shelf Language Models
TLDR
Comprehensive empirical results demonstrate that REFLECTIVE DECODING outperforms strong unsupervised baselines on both paraphrasing and abductive text infilling, significantly narrowing the gap between unsuper supervised and supervised methods.
...
1
2
...