Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering

@inproceedings{Bosselut2021DynamicNK,
  title={Dynamic Neuro-Symbolic Knowledge Graph Construction for Zero-shot Commonsense Question Answering},
  author={Antoine Bosselut and Ronan Le Bras and Yejin Choi},
  booktitle={AAAI},
  year={2021}
}
Understanding narratives requires reasoning about implicit world knowledge related to the causes, effects, and states of situations described in text. At the core of this challenge is how to access contextually relevant knowledge on demand and reason over it. In this paper, we present initial studies toward zero-shot commonsense question answering by formulating the task as inference over dynamically generated commonsense knowledge graphs. In contrast to previous studies for knowledge… 

Figures and Tables from this paper

Commonsense Reasoning for Question Answering with Explanations

TLDR
A latent-variable model is proposed that identifies what type of knowledge from an external knowledge base may be relevant to answering the question, com-putes the commonsense inferences, and predicts the answer, and can learn to provide posterior rationales for why a certain answer was chosen.

ArT: All-round Thinker for Unsupervised Commonsense Question-Answering

TLDR
On all scales of PrLM backbones, ArT shows its brilliant performance and outperforms previous advanced unsupervised models and a reverse thinking mechanism is proposed to conduct bidirectional inferring between cause and effect.

CHARET: Character-centered Approach to Emotion Tracking in Stories

TLDR
This research explores how to leverage current state-of-the-art tools to make inferences about the emotional state of a character in a story as events unfold, in a coherent way, and proposes a character role-labelling approach to emotion tracking that accounts for the semantics of emotions.

A Systematic Investigation of Commonsense Understanding in Large Language Models

TLDR
It is found that the impressive zeroshot performance of large language models is mostly due to existence of dataset bias in the authors' benchmarks, and that leveraging explicit commonsense knowledge does not yield substantial improvement.

Do Language Models Learn Commonsense Knowledge?

Language models (LMs) trained on large amounts of data (e.g., Brown et al., 2020; Patwary et al., 2021) have shown impressive performance on many NLP tasks under the zero-shot and few-shot setup.

Modeling Event-Pair Relations in External Knowledge Graphs for Script Reasoning

TLDR
A knowledge model to learn the inferential relations between events from the whole eventuality KG and then support downstream models by directly capturing the relation between events in a script is proposed and a neural script adapter is presented to extend the knowledge model for inferring the associated relations between an event chain and a subsequent event candidate.

The First Workshop on Commonsense Representation and Reasoning May 27 , 2022

  • 2022

Survey on Applications of Neurosymbolic Artificial Intelligence

—In recent years, the Neurosymbolic framework has attracted a lot of attention in various applications, from recommender systems and information retrieval to healthcare and finance. This success is

Improving Language Model Prompting in Support of Semi-autonomous Task Learning

TLDR
This work describes efforts toward a novel agent capability that can construct cues that result in useful LLM responses for an agent learning a new task.

OneEE: A One-Stage Framework for Fast Overlapping and Nested Event Extraction

TLDR
The model is equipped with an adaptive event fusion module to generate event-aware representations and a distance-aware predictor to integrate relative distance information for word-word relation recognition, which are empiri-cally demonstrated to be effective mechanisms.

References

SHOWING 1-10 OF 45 REFERENCES

Language Models are Unsupervised Multitask Learners

TLDR
It is demonstrated that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText, suggesting a promising path towards building language processing systems which learn to perform tasks from their naturally occurring demonstrations.

Improving Language Understanding by Generative Pre-Training

TLDR
The general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, improving upon the state of the art in 9 out of the 12 tasks studied.

COMET: Commonsense Transformers for Automatic Knowledge Graph Construction

TLDR
This investigation reveals promising results when implicit knowledge from deep pre-trained language models is transferred to generate explicit knowledge in commonsense knowledge graphs, and suggests that using generative commonsense models for automatic commonsense KB completion could soon be a plausible alternative to extractive methods.

Modeling Naive Psychology of Characters in Simple Commonsense Stories

TLDR
A new annotation framework is introduced to explain naive psychology of story characters as fully-specified chains of mental states with respect to motivations and emotional reactions and establishes baseline performance on several new tasks, suggesting avenues for future research.

RoBERTa: A Robustly Optimized BERT Pretraining Approach

TLDR
It is found that BERT was significantly undertrained, and can match or exceed the performance of every model published after it, and the best model achieves state-of-the-art results on GLUE, RACE and SQuAD.

Ranking and Selecting Multi-Hop Knowledge Paths to Better Predict Human Needs

TLDR
A novel method to extract, rank, filter and select multi-hop relation paths from a commonsense knowledge resource to interpret the expression of sentiment in terms of their underlying human needs is presented.

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks.

Social IQA: Commonsense Reasoning about Social Interactions

TLDR
It is established that Social IQa, the first large-scale benchmark for commonsense reasoning about social situations, is challenging for existing question-answering models based on pretrained language models, compared to human performance (>20% gap).

ATOMIC: An Atlas of Machine Commonsense for If-Then Reasoning

TLDR
Experimental results demonstrate that multitask models that incorporate the hierarchical structure of if-then relation types lead to more accurate inference compared to models trained in isolation, as measured by both automatic and human evaluation.

Automated Storytelling via Causal, Commonsense Plot Ordering

TLDR
C2PO is demonstrated, an approach to narrative generation that operationalizes the concept of soft causal relations as causal relations inferred from commonsense reasoning through Causal, Commonsense Plot Ordering.