# Unsupervised Commonsense Question Answering with Self-Talk

@article{Shwartz2020UnsupervisedCQ,
title={Unsupervised Commonsense Question Answering with Self-Talk},
author={Vered Shwartz and Peter West and Ronan Le Bras and Chandra Bhagavatula and Yejin Choi},
journal={ArXiv},
year={2020},
volume={abs/2004.05483}
}
• Published 11 April 2020
• Computer Science
• ArXiv
Natural language understanding involves reading between the lines with implicit background knowledge. Current systems either rely on pre-trained language models as the sole implicit source of world knowledge, or resort to external knowledge bases (KBs) to incorporate additional relevant knowledge. We propose an unsupervised framework based on \emph{self-talk} as a novel alternative to multiple-choice commonsense tasks. Inspired by inquiry-based discovery learning (Bruner, 1961), our approach…
105 Citations

## Figures and Tables from this paper

### Bipolar Radiofrequency Ablation of Benign Symptomatic Thyroid Nodules: Initial Experience.

• Medicine
RoFo : Fortschritte auf dem Gebiete der Rontgenstrahlen und der Nuklearmedizin
• 2016
Bipolar RFA is a safe and effective treatment option for symptomatic benign thyroid nodules andUltrasound imaging allows guidance during bipolar radiofrequency ablation, and scintigraphy is able to detect the ablation area of autonomous adenomas.

### Does Post-Surgery Placement of Rectus Sheath Block Analgesia Alter the Oxidative Stress Biomarker 8-OHdG Concentrations: A Randomised Trial of Patients with Cancer and Benign Disease.

• Medicine
Cancer genomics & proteomics
• 2016
The results suggest that the placement of RSB analgesia does not significantly alter the oxidative stress marker 8-OHdG concentrations in patients with benign disease or cancer patients.

### Comprehension Based Question Answering using Bloom’s Taxonomy

• Computer Science
REPL4NLP
• 2021
This work uses Bloom’s Taxonomy to provide proximal context that helps the model answer questions by being relevant to those questions, and shows targeting context in this manner improves performance across 4 popular common sense question answer datasets.

### Answer-level Calibration for Free-form Multiple Choice Question Answering

This work presents ALC (Answer-Level Calibration), where the main suggestion is to model context-independent biases in terms of the probability of a choice without the associated context and to subsequently remove it using an unsupervised estimate of similarity with the full context.

### Improving Unsupervised Commonsense Reasoning Using Knowledge-Enabled Natural Language Inference

• Computer Science
EMNLP
• 2021
This work shows the effectiveness of using a common framework, Natural Language Inference (NLI), to solve diverse commonsense reasoning tasks, by leveraging transfer learning from large NLI datasets, and injecting crucial knowledge from commonsense sources such as ATOMIC 2020 and ConceptNet.

### A Systematic Investigation of Commonsense Understanding in Large Language Models

• Computer Science
ArXiv
• 2021
It is found that the impressive zeroshot performance of large language models is mostly due to existence of dataset bias in the authors' benchmarks, and that leveraging explicit commonsense knowledge does not yield substantial improvement.

### Do Language Models Learn Commonsense Knowledge?

• Psychology
• 2021
Language models (LMs) trained on large amounts of data (e.g., Brown et al., 2020; Patwary et al., 2021) have shown impressive performance on many NLP tasks under the zero-shot and few-shot setup.

### Think Before You Speak: Using Self-talk to Generate Implicit Commonsense Knowledge for Response Generation

• Computer Science
ArXiv
• 2021
This paper presents a self-talk approach that first generates the implicit commonsense knowledge and then generates response by referencing the externalized knowledge, all using one generative model.

### Prompting Contrastive Explanations for Commonsense Reasoning Tasks

• Computer Science
FINDINGS
• 2021
Inspired by the contrastive nature of human explanations, large pretrained language models are used to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer.

### minicons: Enabling Flexible Behavioral and Representational Analyses of Transformer Language Models

The minicons library is described and applied to two motivating case studies: One focusing on the learning dynamics of the BERT architecture on relative grammatical judgments, and the other on benchmarking 23 different LMs on zero-shot abductive reasoning.

## References

SHOWING 1-10 OF 76 REFERENCES

### Urinary bladder hyporeflexia and reduced pain-related behaviour in P2X 3-deficient mice

• Biology
Nature
• 2000
P2X3 is critical for peripheral pain responses and afferent pathways controlling urinary bladder volume reflexes and may have therapeutic potential in the treatment of disorders of urine storage and voiding such as overactive bladder.

### Re-engineering Online Documentation: Designing Examples-based Online Support Systems

• Computer Science
• 1999
It is argued that three developments in the design and evaluation of online support systems are reshaping traditional design efforts, and borrowing from principles of re-engineering, an alternative user-centered approach to online documentation design is outlined that draws on the metaphor of information as tool.

### Breaking Spectrum Gridlock With Cognitive Radios: An Information Theoretic Perspective

• Computer Science
Proceedings of the IEEE
• 2009
This information-theoretic survey provides guidelines for the spectral efficiency gains possible through cognitive radios, as well as practical design ideas to mitigate the coexistence challenges in today's crowded spectrum.

### Solving forward-backward stochastic differential equations explicitly — a four step scheme

• Mathematics
• 1994
SummaryIn this paper we investigate the nature of the adapted solutions to a class of forward-backward stochastic differential equations (SDEs for short) in which the forward equation is

### XLNet: Generalized Autoregressive Pretraining for Language Understanding

• Computer Science
NeurIPS
• 2019
XLNet is proposed, a generalized autoregressive pretraining method that enables learning bidirectional contexts by maximizing the expected likelihood over all permutations of the factorization order and overcomes the limitations of BERT thanks to its autore progressive formulation.

### BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

• Computer Science
NAACL
• 2019
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.

### Interpretation of Natural Language Rules in Conversational Machine Reading

• Computer Science
EMNLP
• 2018
This paper formalise this task and develops a crowd-sourcing strategy to collect 37k task instances based on real-world rules and crowd-generated questions and scenarios to assess its difficulty by evaluating the performance of rule-based and machine-learning baselines.