Unsupervised Commonsense Question Answering with Self-Talk

@article{Shwartz2020UnsupervisedCQ,
  title={Unsupervised Commonsense Question Answering with Self-Talk},
  author={Vered Shwartz and Peter West and Ronan Le Bras and Chandra Bhagavatula and Yejin Choi},
  journal={ArXiv},
  year={2020},
  volume={abs/2004.05483}
}
Natural language understanding involves reading between the lines with implicit background knowledge. Current systems either rely on pre-trained language models as the sole implicit source of world knowledge, or resort to external knowledge bases (KBs) to incorporate additional relevant knowledge. We propose an unsupervised framework based on \emph{self-talk} as a novel alternative to multiple-choice commonsense tasks. Inspired by inquiry-based discovery learning (Bruner, 1961), our approach… 

Figures and Tables from this paper

Commonsense Reasoning for Natural Language Processing
TLDR
This tutorial organizes this tutorial to provide researchers with the critical foundations and recent advances in commonsense representation and reasoning, in the hopes of casting a brighter light on this promising area of future research.
Self-Supervised Knowledge Triplet Learning for Zero-shot Question Answering
TLDR
This work proposes Knowledge Triplet Learning (KTL), a self-supervised task over knowledge graphs that proposes heuristics to create synthetic graphs for commonsense and scientific knowledge and shows considerable improvements over large pre-trained transformer models.
Comprehension Based Question Answering using Bloom’s Taxonomy
TLDR
This work uses Bloom’s Taxonomy to provide proximal context that helps the model answer questions by being relevant to those questions, and shows targeting context in this manner improves performance across 4 popular common sense question answer datasets.
Answer-level Calibration for Free-form Multiple Choice Question Answering
Pre-trained language models have recently shown that training on large corpora using the language modeling objective enables few-shot and zero-shot capabilities on a variety of NLP tasks, including
A Systematic Investigation of Commonsense Understanding in Large Language Models
TLDR
It is found that the impressive zeroshot performance of large language models is mostly due to existence of dataset bias in the authors' benchmarks, and that leveraging explicit commonsense knowledge does not yield substantial improvement.
Do Language Models Learn Commonsense Knowledge?
Language models (LMs) trained on large amounts of data (e.g., Brown et al., 2020; Patwary et al., 2021) have shown impressive performance on many NLP tasks under the zero-shot and few-shot setup.
Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference
TLDR
This work shows the effectiveness of using a common framework, Natural Language Inference (NLI), to solve diverse commonsense reasoning tasks, by leveraging transfer learning from large NLI datasets, and injecting crucial knowledge from commonsense sources such as ATOMIC 2020 and ConceptNet.
Prompting Contrastive Explanations for Commonsense Reasoning Tasks
TLDR
Inspired by the contrastive nature of human explanations, large pretrained language models are used to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer.
Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation
TLDR
This paper presents a self-talk approach that first generates the implicit commonsense knowledge and then generates response by referencing the externalized knowledge, all using one generative model.
Zero-shot Commonsense Question Answering with Cloze Translation and Consistency Optimization
TLDR
Four translation methods that can translate natural questions into cloze-style sentences to better solicit commonsense knowledge from language models are investigated, including a syntactic-based model, an unsupervised neural model, and two supervised neural models.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 76 REFERENCES
Dynamic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
TLDR
Empirical results on the SocialIQa and StoryCommonsense datasets in a zero-shot setting demonstrate that using commonsense knowledge models to dynamically construct and reason over knowledge graphs achieves performance boosts over pre-trained language models and usingknowledge models to directly evaluate answers.
Language Models as Knowledge Bases?
TLDR
An in-depth analysis of the relational knowledge already present (without fine-tuning) in a wide range of state-of-the-art pretrained language models finds that BERT contains relational knowledge competitive with traditional NLP methods that have some access to oracle knowledge.
Commonsense for Generative Multi-Hop Question Answering Tasks
TLDR
This work focuses on a more challenging multi-hop generative task (NarrativeQA), which requires the model to reason, gather, and synthesize disjoint pieces of information within the context to generate an answer.
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
TLDR
This work presents CommonsenseQA: a challenging new dataset for commonsense question answering, which extracts from ConceptNet multiple target concepts that have the same semantic relation to a single source concept.
Incorporating Relation Knowledge into Commonsense Reading Comprehension with Multi-task Learning
TLDR
This paper designs two auxiliary relation-aware tasks to predict if there exists any commonsense relation and what is the relation type be-tween two words, in order to better model the interactions between document and candidate answer option.
Language Models are Unsupervised Multitask Learners
TLDR
It is demonstrated that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText, suggesting a promising path towards building language processing systems which learn to perform tasks from their naturally occurring demonstrations.
Commonsense Knowledge Mining from Pretrained Models
TLDR
This work develops a method for generating commonsense knowledge using a large, pre-trained bidirectional language model that can be used to rank a triple’s validity by the estimated pointwise mutual information between the two entities.
Barack’s Wife Hillary: Using Knowledge Graphs for Fact-Aware Language Modeling
TLDR
This work introduces the knowledge graph language model (KGLM), a neural language model with mechanisms for selecting and copying facts from a knowledge graph that are relevant to the context that enable the model to render information it has never seen before, as well as generate out-of-vocabulary tokens.
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
TLDR
This work collects human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation framework.
Improving Language Understanding by Generative Pre-Training
TLDR
The general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, improving upon the state of the art in 9 out of the 12 tasks studied.
...
1
2
3
4
5
...