A Knowledge Hunting Framework for Common Sense Reasoning

@inproceedings{Emami2018AKH,
  title={A Knowledge Hunting Framework for Common Sense Reasoning},
  author={Ali Emami and Noelia De La Cruz and Adam Trischler and Kaheer Suleman and Jackie Chi Kit Cheung},
  booktitle={Conference on Empirical Methods in Natural Language Processing},
  year={2018}
}
We introduce an automatic system that achieves state-of-the-art results on the Winograd Schema Challenge (WSC), a common sense reasoning task that requires diverse, complex forms of inference and knowledge. [] Key Method Our method uses a knowledge hunting module to gather text from the web, which serves as evidence for candidate problem resolutions.
[PDF] Semantic Reader
32 Citations
Highly Influential Citations
5
Background Citations
16
Methods Citations
10
Results Citations
2

Tables from this paper

32 Citations

Combining Knowledge Hunting and Neural Language Models to Solve the Winograd Schema Challenge

This work builds-up on the language model based methods and augment them with a commonsense knowledge hunting (using automatic extraction from text) module and an explicit reasoning module and achieves the state-of-the-art accuracy on the WSC dataset.

Commonsense Knowledge Types Identification and Reasoning for the Winograd Schema Challenge

  • Computer Science
  • 2018
This work performed a comprehensive study of the problems in the Winograd Schema Challenge from the perspective of identifying the types of knowledge which are needed to solve them and defined a logical reasoning algorithm to tackle 10 out of 12 knowledge types.

On the Evaluation of Common-Sense Reasoning in Natural Language Understanding

A case study of the Winograd Schema Challenge is made and a protocol is designed, based on two new measures of instance-level complexity, that both clarifies and qualifies the results of previous work.

Attention Is (not) All You Need for Commonsense Reasoning

A simple re-implementation of BERT for commonsense reasoning is described and it is shown that the attentions produced by BERT can be directly utilized for tasks such as the Pronoun Disambiguation Problem and Winograd Schema Challenge.

The Winograd Schema Challenge: Are You Sure That We Are on the Right Track?

It is argued that most systems developed so far have typically been designed and evaluated without considering the challenge’s purpose, emphasizing the semblance of intelligence rather than understanding human behavior itself.

Exploring Unsupervised Pretraining and Sentence Structure Modelling for Winograd Schema Challenge

It is demonstrated that the leading performance benefits from jointly modelling sentence structures, utilizing knowledge learned from cutting-edge pretraining models, and performing fine-tuning.

A Review of Winograd Schema Challenge Datasets and Approaches

This paper reviews existing Winograd Schema Challenge benchmark datasets and approaches that have been published since its introduction and suggests new approaches that should be considered.

WinoWhy: A Deep Diagnosis of Essential Commonsense Knowledge for Answering Winograd Schema Challenge

This paper presents the first comprehensive categorization of essential commonsense knowledge for answering the Winograd Schema Challenge (WSC), and leverages the collected reasons to develop a new task called WinoWhy, which requires models to distinguish plausible reasons from very similar but wrong reasons for all WSC questions.

Winnowing Knowledge for Multi-choice Question Answering

A novel encoding method is proposed which is able to conduct interception and soft filtering which contributes to the harvesting and absorption of representative information with less interference from noises.

The Defeat of the Winograd Schema Challenge

The history of the Winograd Schema Challenge is reviewed, and a number of AI systems, based on large pre-trained transformer-based language models and fine-tuned on these kinds of problems, achieved better than 90% accuracy.

References

SHOWING 1-10 OF 33 REFERENCES

A Generalized Knowledge Hunting Framework for the Winograd Schema Challenge

This work introduces an automatic system that performs well on two common-sense reasoning tasks, the Winograd Schema Challenge (WSC) and the Choice of Plausible Alternatives (COPA), and is competitive with the state-of-the-art on COPA, demonstrating its general applicability.

Towards Addressing the Winograd Schema Challenge - Building and Using a Semantic Parser and a Knowledge Hunting Module

This paper presents an approach that identifies the knowledge needed to answer a challenge question, hunts down that knowledge from text repositories, and then reasons with machines to come up with the answer.

ConceptNet — A Practical Commonsense Reasoning Tool-Kit

ConceptNet is a freely available commonsense knowledge base and natural-language-processing tool-kit which supports many practical textual-reasoning tasks over real-world documents including…

Commonsense Reasoning in a Deeper Way: By Discovering Relations between Predicates

This paper develops a commonsense reasoning framework, which gives a formal definition of this kind of commonsense, and constructs a set of knowledge by extending the predicate set of ConceptNet, and applies information extraction technique to capture them from corpus.

Knowledge Representation for Commonsense Reasoning with Text

It is claimed that there is such a level of knowledge, called naive semantics (NS), which is commonsense knowledge associated with words, that is general and common to many speakers of a natural language and can be represented for use in computational text understanding.

An Example-Based Approach to Difficult Pronoun Resolution

The experimental results show that the existing sentences on the Web indeed contain instances of world knowledge useful for difficult pronoun resolution, and a method for automatically acquiring examples that are similar to Winograd schemas but have less ambiguity is presented.

A Simple Method for Commonsense Reasoning

Key to this method is the use of language models, trained on a massive amount of unlabled data, to score multiple choice questions posed by commonsense reasoning tests, which outperform previous state-of-the-art methods by a large margin.

The Winograd Schema Challenge and Reasoning about Correlation

A framework for reasoning about correlation between sentences is introduced, and it is shown how this framework can be used to justify solutions to some Winograd Schema problems.

Combing Context and Commonsense Knowledge Through Neural Networks for Solving Winograd Schema Problems

A general framework to combine context and commonsense knowledge for solving the Winograd Schema (WS) and Pronoun Disambiguation Problems (PDP) and two methods to solve the WS and PDP problems are proposed.

Solving Hard Coreference Problems

This paper presents a general coreference resolution system that significantly improves state-of-the-art performance on hard, Winograd-style, pronoun resolution cases, while still performing at the state of the art level on standard coreferenceresolution datasets.