• Corpus ID: 222134165

Paragraph-Level Commonsense Transformers with Recurrent Memory

@inproceedings{Gabriel2021ParagraphLevelCT,
  title={Paragraph-Level Commonsense Transformers with Recurrent Memory},
  author={Saadia Gabriel and Chandra Bhagavatula and Vered Shwartz and Ronan Le Bras and Maxwell Forbes and Yejin Choi},
  booktitle={AAAI},
  year={2021}
}
Human understanding of narrative texts requires making commonsense inferences beyond what is stated in the text explicitly. A recent model, COMeT, can generate such inferences along several dimensions such as pre- and post-conditions, motivations, and mental-states of the participants. However, COMeT was trained on short phrases, and is therefore discourse-agnostic. When presented with each sentence of a multi-sentence narrative, it might generate inferences that are inconsistent with the rest… 

Figures and Tables from this paper

Implicit Premise Generation with Discourse-aware Commonsense Knowledge Models
TLDR
It is demonstrated that encoding discourse-aware commonsense during fine-tuning improves the quality of the generated implicit premises and outperforms all other baselines both in automatic and human evaluations on three different datasets.
Analyzing the Contribution of Commonsense Knowledge Sources for Why-Question Answering
TLDR
There is potential for methods that can automatically select and inject commonsense from relevant sources and a simple ontology of knowledge types is developed and the relative cover026 age of the different models on these categories are analyzed.
CoCoLM: Complex Commonsense Enhanced Language Model with Discourse Relations
TLDR
This paper proposes a general language model named CoCoLM, which through the careful training over a large-scale eventuality knowledge graph ASER, successfully teaches pre-trained language models (i.e., BERT and RoBERTa) rich multi-hop commonsense knowledge among eventualities.
It’s not Rocket Science: Interpreting Figurative Language in Narratives
TLDR
Knowledge-enhanced models are proposed, adopting human strategies for interpreting figurative language types: inferring meaning from the context and relying on the constituent words’ literal meanings, further bridging the gap from human performance.
Learning from Missing Relations: Contrastive Learning with Commonsense Knowledge Graphs for Commonsense Inference
TLDR
This paper focuses on addressing missing relations in commonsense knowledge graphs, and proposes a novel contrastive learning framework called SOLAR, which outperforms the state-of-the-art commonsense transformer on commonsense inference with ConceptNet by 1.84%.
The Power of Reuse: A Multi-Scale Transformer Model for Structural Dynamic Segmentation in Symbolic Music Generation
TLDR
This paper proposes a multi-scale Transformer, which uses coarse-decoder and fine-decoders to model the contexts at the global and section-level, respectively, and demonstrates that the model outperforms the best con-temporary symbolic music generative models.
Towards Teachable Reasoning Systems
TLDR
Generated chains of reasoning show how answers are implied by the system’s own internal beliefs, and are both faithful and truthful, which suggests new opportunities for using language models in an interactive setting where users can inspect, debug, correct, and improve a system‘s performance over time.
Automated Story Generation as Question-Answering
TLDR
A novel approach to automated story generation that treats the problem as one of generative question-answering, and shows that stories generated by the system are 15% more coherent than those generated by BART fine-tuned on a story corpus to generate backward from a given ending.
Automatic Story Generation: Challenges and Attempts
TLDR
This paper analyzes works in story generation that utilize machine learning approaches to address story generation controllability, incorporate commonsense knowledge, infer reasonable character actions, and generate creative language.
COINS: Dynamically Generating COntextualized Inference Rules for Narrative Story Completion
TLDR
Coins, a recursive inference framework that iteratively reads context sentences, dynamically generates contextualized inference rules, encodes them, and uses them to guide task-specific output generation, generates better story sentences than SOTA baselines, especially in terms of coherence.
...
1
2
...

References

SHOWING 1-10 OF 49 REFERENCES
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
TLDR
This paper introduces Cosmos QA, a large-scale dataset of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions, and proposes a new architecture that improves over the competitive baselines.
A Knowledge-Enhanced Pretraining Model for Commonsense Story Generation
TLDR
A knowledge-enhanced pretraining model to utilize commonsense knowledge from external knowledge bases to generate reasonable stories that can generate more reasonable stories than state-of-the-art baselines, particularly in terms of logic and global coherence.
A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories
TLDR
A new framework for evaluating story understanding and script learning: the `Story Cloze Test’, which requires a system to choose the correct ending to a four-sentence story, and a new corpus of 50k five- Sentence commonsense stories, ROCStories, to enable this evaluation.
Ordinal Common-sense Inference
TLDR
This work describes a framework for extracting common-sense knowledge from corpora, which is then used to construct a dataset for this ordinal entailment task, and annotates subsets of previously established datasets via the ordinal annotation protocol in order to analyze the distinctions between these and what is constructed.
Unsupervised Learning of Narrative Event Chains
TLDR
A three step process to learning narrative event chains using unsupervised distributional methods to learn narrative relations between events sharing coreferring arguments and introduces two evaluations: the narrative cloze to evaluate event relatedness, and an order coherence task to evaluate narrative order.
ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning
TLDR
Experimental results demonstrate that multitask models that incorporate the hierarchical structure of if-then relation types lead to more accurate inference compared to models trained in isolation, as measured by both automatic and human evaluation.
Understanding Stories with Large-Scale Common Sense
TLDR
The Aspire system is implemented, an application of large-scale commonsense knowledge to story understanding that extends Genesis, a rule-based story understanding system, with tens of thousands of goalrelated assertions from the commonsense semantic network ConceptNet.
Modeling Naive Psychology of Characters in Simple Commonsense Stories
TLDR
A new annotation framework is introduced to explain naive psychology of story characters as fully-specified chains of mental states with respect to motivations and emotional reactions and establishes baseline performance on several new tasks, suggesting avenues for future research.
COMET: Commonsense Transformers for Automatic Knowledge Graph Construction
TLDR
This investigation reveals promising results when implicit knowledge from deep pre-trained language models is transferred to generate explicit knowledge in commonsense knowledge graphs, and suggests that using generative commonsense models for automatic commonsense KB completion could soon be a plausible alternative to extractive methods.
What Happens Next? Event Prediction Using a Compositional Neural Network Model
TLDR
It is found that recent work on learning vector-space embeddings to capture word meaning can be effectively applied to this task, including simple incorporation of a verb's arguments in the representation by vector addition.
...
1
2
3
4
5
...