CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning

  title={CommonGen: A Constrained Text Generation Challenge for Generative Commonsense Reasoning},
  author={Bill Yuchen Lin and Minghan Shen and Wangchunshu Zhou and Pei Zhou and Chandra Bhagavatula and Yejin Choi and Xiang Ren},
Recently, large-scale pre-trained language models have demonstrated impressive performance on several commonsense-reasoning benchmark datasets. However, building machines with commonsense to compose realistically plausible sentences remains challenging. In this paper, we present a constrained text generation task, CommonGen associated with a benchmark dataset, to explicitly test machines for the ability of generative commonsense reasoning. Given a set of common concepts (e.g., dog, frisbee… 

MVP: Multi-task Supervised Pre-training for Natural Language Generation

This work proposes M ulti-task super V ised P re-training ( MVP) for natural language generation, and collects a labeled pre-training corpus from 45 datasets over seven generation tasks to pre-train the text generation model MVP.

NeuroLogic A*esque Decoding: Constrained Text Generation with Lookahead Heuristics

NeuroLogic A*esque is proposed, a decoding algorithm that incorporates heuristic estimates of future cost that develops lookahead heuristics that are efficient for large-scale language models, making this method a drop-in replacement for common techniques such as beam search and top-k sampling.

A Dog Is Passing Over The Jet? A Text-Generation Dataset for Korean Commonsense Reasoning and Evaluation

This work proposes a text-generation dataset for Korean generative commonsense reasoning and language model evaluation, and presents an in-depth analysis of the generation results of language models with various evaluation metrics along with human-annotated scores.

Controllable Text Generation with Neurally-Decomposed Oracle

A general and efficient framework to control auto-regressive generation models with NeurAlly-Decomposed Oracle (NADO) guides the base model towards the given oracle while maintaining high generation quality.

Revisiting Generative Commonsense Reasoning: A Pre-Ordering Approach

It is argued that PTM’s inherent ability for generative commonsense reasoning is underestimated due to the order-agnostic property of its input, and proposed a pre-ordering approach to elaborately manipulate the order of the given concepts before generation.

Fine-Grained Controllable Text Generation Using Non-Residual Prompting

This work proposes an encoder-decoder architecture that enables intermediate text prompts at arbitrary time steps, and proposes a resource-efficient method for converting a pre-trained CLM into this architecture, and demonstrates its potential on various experiments, including the novel task of contextualized word inclusion.

Task2Dial: A Novel Task and Dataset for Commonsense-enhanced Task-based Dialogue Grounded in Documents

The Task2Dial dataset is described, a novel dataset of document-grounded task-based dialogues, where an Information Giver provides instructions (by consulting a document) to an Information Follower, so that the latter can successfully complete the task.

KGR^4: Retrieval, Retrospect, Refine and Rethink for Commonsense Generation

A novel Knowledge-enhanced Commonsense Generation framework, termed KGR4, consisting of four stages: Retrieval, Retrospect, Refine, Rethink, which selects the output sentence from candidate sentences produced by generators with different hyper-parameters.

Contextualized Scene Imagination for Generative Commonsense Reasoning

An Imagine-and-Verbalize (I&V) method is proposed, which learns to imagine a relational scene knowledge graph (SKG) with relations between the input concepts, and leverage the SKG as a constraint when generating a plausible scene description.

Improving Abstractive Summarization with Commonsense Knowledge

Two methods to add commonsense reasoning skills and knowledge into abstractive summarization models are introduced and human evaluation results suggest that summaries generated by these methods are more realistic and have fewer commonsensical errors.



Conflict and context in peer relations.

Hematogenous Salmonella Typhi Osteomyelitis of the Radius: A Case Report

Normal function was restored after radical debridement, intravenous antibiotics, and delayed tricortical iliac crest bone grafting of the segmental defect.

Breaking Spectrum Gridlock With Cognitive Radios: An Information Theoretic Perspective

This information-theoretic survey provides guidelines for the spectral efficiency gains possible through cognitive radios, as well as practical design ideas to mitigate the coexistence challenges in today's crowded spectrum.

Unified Language Model Pre-training for Natural Language Understanding and Generation

A new Unified pre-trained Language Model (UniLM) that can be fine-tuned for both natural language understanding and generation tasks that compares favorably with BERT on the GLUE benchmark, and the SQuAD 2.0 and CoQA question answering tasks.

SPICE: Semantic Propositional Image Caption Evaluation

There is considerable interest in the task of automatically generating image captions. However, evaluation is challenging. Existing automatic evaluation metrics are primarily sensitive to n-gram

Lexically Constrained Neural Machine Translation with Levenshtein Transformer

A simple and effective algorithm for incorporating lexical constraints in neural machine translation that leverages the flexibility and speed of a recently proposed Levenshtein Transformer model and injects terminology constraints at inference time without any impact on decoding speed.

UniLMv2: Pseudo-Masked Language Models for Unified Language Model Pre-Training

The experiments show that the unified language models pre-trained using PMLM achieve new state-of-the-art results on a wide range of natural language understanding and generation tasks across several widely used benchmarks.

BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension

BART is presented, a denoising autoencoder for pretraining sequence-to-sequence models, which matches the performance of RoBERTa on GLUE and SQuAD, and achieves new state-of-the-art results on a range of abstractive dialogue, question answering, and summarization tasks.

Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

This systematic study compares pre-training objectives, architectures, unlabeled datasets, transfer approaches, and other factors on dozens of language understanding tasks and achieves state-of-the-art results on many benchmarks covering summarization, question answering, text classification, and more.

Structural scaling and functional design of the cercal wind-receptor hairs of cricket

Abstract We have estimated the intrinsic mechanical parameters of cricket cercal wind-receptor hairs. The hairs were modeled as an inverted pendulum, and mechanical parameters of the equation of