CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge

@inproceedings{Talmor2019CommonsenseQAAQ,
  title={CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge},
  author={Alon Talmor and Jonathan Herzig and Nicholas Lourie and Jonathan Berant},
  booktitle={NAACL},
  year={2019}
}
When answering a question, people often draw upon their rich world knowledge in addition to some task-specific context. [...] Key Result Our best baseline, the OpenAI GPT (Radford et al., 2018), obtains 54.8% accuracy, well below human performance, which is 95.3%.Expand
Incorporating Domain Knowledge and Semantic Information into Language Models for Commonsense Question Answering
TLDR
This work proposes an approach to incorporate domain knowledge and semantic information into language model based approaches for better understanding the related commonsense knowledge and utilizes Semantic Role Labeling to enable the system to gain a better understanding of relations among relevant entities. Expand
Improving Commonsense Question Answering by Graph-based Iterative Retrieval over Multiple Knowledge Sources
TLDR
A novel question-answering method by integrating multiple knowledge sources, i.e. ConceptNet, Wikipedia, and the Cambridge Dictionary, to boost the performance and introduces a novel graph-based iterative knowledge retrieval module, which iteratively retrieves concepts and entities related to the given question and its choices from multipleknowledge sources. Expand
Reasoning Paths Generation for Commonsense Question Answering
  • Peifeng Wang
  • 2019
Commonsense question answering (QA) requires a model to acquire some necessary background knowledge about how the world operates and people interact with each others. A large number of works haveExpand
Knowledge-driven Self-supervision for Zero-shot Commonsense Question Answering
TLDR
A novel neuro-symbolic framework for zero-shot question answering across commonsense tasks is proposed and it is shown that, while an individual knowledge graph is better suited for specific tasks, a global knowledge graph brings consistent gains across different tasks. Expand
Learning to Generate Multi-Hop Knowledge Paths for Commonsense Question Answering
  • 2020
Commonsense question answering (QA) requires a model of general background knowledge about how the world operates and how people interact with each other before reasoning. Prior works focus primarilyExpand
Benchmarking Knowledge-Enhanced Commonsense Question Answering via Knowledge-to-Text Transformation
TLDR
This work benchmarked knowledge-enhanced CQA by conducting extensive experiments on multiple standard C QA datasets using a simple and effective knowledgeto-text transformation framework and shows that context-sensitive knowledge selection, heterogeneous knowledge exploitation, and commonsense-rich language models are promising CZA directions. Expand
A Semantic-based Method for Unsupervised Commonsense Question Answering
TLDR
A novel SEmanticbased Question Answering method (SEQA) for unsupervised commonsense question answering that first generates a set of plausible answers with generative models, and then uses these plausible answers to select the correct choice by considering the semantic similarity between each plausible answer and each choice. Expand
Commonsense Question Answering: A Survey
  • Jun Yan
  • 2019
When humans use their languages to communicate with each other, they often rely on broad implicit assumptions. Humans learn and use this kind of assumptions in everyday life, which makes theirExpand
Towards Generative Commonsense Reasoning: A Concept Paper
  • Bill Yuchen Lin
  • 2019
In this concept paper, we first review recent advances in machine commonsense reasoning and further investigate their potential connections with natural language generation. Current research effortsExpand
COMMONGEN: Towards Generative Commonsense Reasoning via A Constrained Text Generation Challenge
Given a set of common concepts like “{apple (noun), pick (verb), tree (noun)}”, humans find it easy to write a sentence describing a grammatical and logically coherent scenario that covers theseExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 44 REFERENCES
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
TLDR
It is shown that, in comparison to other recently introduced large-scale datasets, TriviaQA has relatively complex, compositional questions, has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and requires more cross sentence reasoning to find answers. Expand
Deep Learning for Answer Sentence Selection
TLDR
This work proposes a novel approach to solving the answer sentence selection task via means of distributed representations, and learns to match questions with answers by considering their semantic encoding. Expand
Think you have Solved Question Answering? Try ARC, the AI2 Reasoning Challenge
TLDR
A new question set, text corpus, and baselines assembled to encourage AI research in advanced question answering constitute the AI2 Reasoning Challenge (ARC), which requires far more powerful knowledge and reasoning than previous challenges such as SQuAD or SNLI. Expand
SQuAD: 100,000+ Questions for Machine Comprehension of Text
TLDR
A strong logistic regression model is built, which achieves an F1 score of 51.0%, a significant improvement over a simple baseline (20%). Expand
A Simple Method for Commonsense Reasoning
TLDR
Key to this method is the use of language models, trained on a massive amount of unlabled data, to score multiple choice questions posed by commonsense reasoning tests, which outperform previous state-of-the-art methods by a large margin. Expand
A Corpus and Cloze Evaluation for Deeper Understanding of Commonsense Stories
TLDR
A new framework for evaluating story understanding and script learning: the `Story Cloze Test’, which requires a system to choose the correct ending to a four-sentence story, and a new corpus of 50k five- Sentence commonsense stories, ROCStories, to enable this evaluation. Expand
MCScript: A Novel Dataset for Assessing Machine Comprehension Using Script Knowledge
TLDR
A large dataset of narrative texts and questions about these texts, intended to be used in a machine comprehension task that requires reasoning using commonsense knowledge, and shows that the mode of data collection via crowdsourcing results in a substantial amount of inference questions. Expand
Improving Language Understanding by Generative Pre-Training
TLDR
The general task-agnostic model outperforms discriminatively trained models that use architectures specifically crafted for each task, significantly improving upon the state of the art in 9 out of the 12 tasks studied. Expand
Bidirectional Attention Flow for Machine Comprehension
TLDR
The BIDAF network is introduced, a multi-stage hierarchical process that represents the context at different levels of granularity and uses bi-directional attention flow mechanism to obtain a query-aware context representation without early summarization. Expand
Can a Suit of Armor Conduct Electricity? A New Dataset for Open Book Question Answering
TLDR
A new kind of question answering dataset, OpenBookQA, modeled after open book exams for assessing human understanding of a subject, and oracle experiments designed to circumvent the knowledge retrieval bottleneck demonstrate the value of both the open book and additional facts. Expand
...
1
2
3
4
5
...