Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge

@inproceedings{Mihaylov2018KnowledgeableRE,
  title={Knowledgeable Reader: Enhancing Cloze-Style Reading Comprehension with External Commonsense Knowledge},
  author={Todor Mihaylov and Anette Frank},
  booktitle={ACL},
  year={2018}
}
We introduce a neural reading comprehension model that integrates external commonsense knowledge, encoded as a key-value memory, in a cloze-style setting. Instead of relying only on document-to-question interaction or discrete features as in prior work, our model attends to relevant external knowledge and combines this knowledge with the context representation before inferring the answer. This allows the model to attract and imply knowledge from an external knowledge source that is not… Expand
Incorporating External Knowledge into Machine Reading for Generative Question Answering
TLDR
A new neural model, Knowledge-Enriched Answer Generator (KEAG), which is able to compose a natural answer by exploiting and aggregating evidence from all four information sources available: question, passage, vocabulary and knowledge. Expand
Enhancing Transformer-based language models with commonsense representations for knowledge-driven machine comprehension
TLDR
This work presents three simple yet effective injection methods integrated into the structure of TrLMs to fine-tune downstream knowledge-driven MRC tasks with off-the-shelf commonsense representations and introduces a mask mechanism for a token-level multi-hop relationship searching to filter external knowledge. Expand
Explicit Utilization of General Knowledge in Machine Reading Comprehension
TLDR
An end-to-end MRC model named as Knowledge Aided Reader (KAR), which explicitly uses the above extracted general knowledge to assist its attention mechanisms and is comparable in performance with the state-of-the-art MRC models, and significantly more robust to noise than them. Expand
Learning to Generate Multi-Hop Knowledge Paths for Commonsense Question Answering
  • 2020
Commonsense question answering (QA) requires a model of general background knowledge about how the world operates and how people interact with each other before reasoning. Prior works focus primarilyExpand
Contextualized Representations Using Textual Encyclopedic Knowledge
TLDR
It is shown that integrating background knowledge from text is effective for tasks focusing on factual reasoning and allows direct reuse of powerful pretrained BERT-style encoders and knowledge integration can be further improved with suitable pretraining via a self-supervised masked language model objective over words in background-augmented input text. Expand
Incorporating Structured Commonsense Knowledge in Story Completion
TLDR
Experiments show that the model outperforms state-of-the-art approaches on a public dataset, ROCStory Cloze Task, and the performance gain from adding the additional commonsense knowledge is significant. Expand
Towards Knowledge Enhanced Language Model for Machine Reading Comprehension
TLDR
A novel deep model KCF-NET is proposed, which incorporates knowledge graph representations with context as the basis for predicting answers by leveraging capsule network to encode the intrinsic spatial relationship in triples of KG. Expand
Reference Knowledgeable Network for Machine Reading Comprehension
TLDR
This work proposes a novel reference-based knowledge enhancement model based on span extraction called Reference Knowledgeable Network (RekNet), which simulates human reading strategy to refine critical information from the passage and quote external knowledge in necessity. Expand
Coarse-To-Careful: Seeking Semantic-Related Knowledge for Open-Domain Commonsense Question Answering
TLDR
A semantic-driven knowledge-aware QA framework that controls the knowledge injection in a coarse-to-careful fashion, and a semantic-aware knowledge fetching module that engages structural knowledge information and fuses proper knowledge according to the careful semantic of questions in a hierarchical way are proposed. Expand
Dynamic Knowledge Graph Construction for Zero-shot Commonsense Question Answering
TLDR
Empirical results on the SocialIQa and StoryCommonsense datasets in a zero-shot setting demonstrate that using commonsense knowledge models to dynamically construct and reason over knowledge graphs achieves performance boosts over pre-trained language models and usingknowledge models to directly evaluate answers. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 43 REFERENCES
Dynamic Integration of Background Knowledge in Neural NLU Systems
TLDR
A new reading architecture for the dynamic integration of explicit background knowledge in NLU models is developed by developing a new task-agnostic reading module that provides refined word representations to a task-specific NLU architecture by processing background knowledge from static corpora in the form of free-text statements. Expand
Gated-Attention Readers for Text Comprehension
TLDR
The model, the Gated-Attention (GA) Reader, integrates a multi-hop architecture with a novel attention mechanism, which is based on multiplicative interactions between the query embedding and the intermediate states of a recurrent neural network document reader, which enables the reader to build query-specific representations of tokens in the document for accurate answer selection. Expand
Text Understanding with the Attention Sum Reader Network
TLDR
A new, simple model is presented that uses attention to directly pick the answer from the context as opposed to computing the answer using a blended representation of words in the document as is usual in similar models, making the model particularly suitable for question-answering problems where the answer is a single word from the document. Expand
Open Mind Common Sense: Knowledge Acquisition from the General Public
TLDR
The first fielded system, which enabled the construction of a 450,000 assertion commonsense knowledge base, is described and evaluated and how the second-generation system addresses weaknesses discovered in the first. Expand
World Knowledge for Reading Comprehension: Rare Entity Prediction with Hierarchical LSTMs Using External Descriptions
TLDR
A task of rare entity prediction: given a web document with several entities removed, models are tasked with predicting the correct missing entities conditioned on the document context and the lexical resources and two recurrent neural network architectures are proposed which make use of external knowledge in the form of entity descriptions. Expand
Attention-over-Attention Neural Networks for Reading Comprehension
TLDR
Experimental results show that the proposed attention-over-attention model significantly outperforms various state-of-the-art systems by a large margin in public datasets, such as CNN and Children's Book Test datasets. Expand
MCTest: A Challenge Dataset for the Open-Domain Machine Comprehension of Text
TLDR
MCTest is presented, a freely available set of stories and associated questions intended for research on the machine comprehension of text that requires machines to answer multiple-choice reading comprehension questions about fictional stories, directly tackling the high-level goal of open-domain machine comprehension. Expand
Key-Value Memory Networks for Directly Reading Documents
TLDR
This work introduces a new method, Key-Value Memory Networks, that makes reading documents more viable by utilizing different encodings in the addressing and output stages of the memory read operation. Expand
Gated Self-Matching Networks for Reading Comprehension and Question Answering
TLDR
The gated self-matching networks for reading comprehension style question answering, which aims to answer questions from a given passage, are presented and holds the first place on the SQuAD leaderboard for both single and ensemble model. Expand
TriviaQA: A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension
TLDR
It is shown that, in comparison to other recently introduced large-scale datasets, TriviaQA has relatively complex, compositional questions, has considerable syntactic and lexical variability between questions and corresponding answer-evidence sentences, and requires more cross sentence reasoning to find answers. Expand
...
1
2
3
4
5
...