Corpus ID: 215745173

Explaining Question Answering Models through Text Generation

@article{Latcinnik2020ExplainingQA,
  title={Explaining Question Answering Models through Text Generation},
  author={Veronica Latcinnik and Jonathan Berant},
  journal={ArXiv},
  year={2020},
  volume={abs/2004.05569}
}
Large pre-trained language models (LMs) have been shown to perform surprisingly well when fine-tuned on tasks that require commonsense and world knowledge. However, in end-to-end architectures, it is difficult to explain what is the knowledge in the LM that allows it to make a correct prediction. In this work, we propose a model for multi-choice question answering, where a LM-based generator generates a textual hypothesis that is later used by a classifier to answer the question. The hypothesis… Expand
Are Training Resources Insufficient? Predict First Then Explain!
TLDR
It is argued that the predict-then-explain (PtE) architecture is a more efficient approach in terms of the modelling perspective, and it is shown that the PtE structure is the most data-efficient approach when explanation data are lacking. Expand
Summarize-then-Answer: Generating Concise Explanations for Multi-hop Reading Comprehension
TLDR
This work proposes an abstractive approach to generate a question-focused, abstractive summary of input paragraphs and then feed it to an RC system, which can generate more compact explanations than an extractive explainer with limited supervision while maintaining sufficiency. Expand
Why do you think that? Exploring faithful sentence–level rationales without supervision
TLDR
This work proposes a differentiable training–framework to create models which output faithful rationales on a sentence level, by solely applying supervision on the target task, and exploits the transparent decision–making process of these models to prefer selecting the correct rationales by applying direct supervision, thereby boosting the performance on the rationale–level. Expand
Flexible Generation of Natural Language Deductions
TLDR
PARAPATTERN is a method for building models to generate deductive inferences from diverse natural language inputs without direct human supervision, and achieves 85% validity on examples of the ‘substitution’ operation from EntailmentBank without the use of any in-domain training data. Expand
Flexible Operations for Natural Language Deduction
TLDR
This paper uses a BART-based model to generate the result of applying a particular logical operation to one or more premise statements, and has a largely automated pipeline for scraping and constructing suitable training examples from Wikipedia, which are then paraphrased to give the models the ability to handle lexical variation. Expand
Learning to Rationalize for Nonmonotonic Reasoning with Distant Supervision
TLDR
This paper investigates multiple ways to automatically generate rationales using pre-trained language models, neural knowledge models, and distant supervision from related tasks, and trains generative models capable of composing explanatory rationales for unseen instances. Expand
Prompting Contrastive Explanations for Commonsense Reasoning Tasks
TLDR
Inspired by the contrastive nature of human explanations, large pretrained language models are used to complete explanation prompts which contrast alternatives according to the key attribute(s) required to justify the correct answer. Expand
You Can Do Better! If You Elaborate the Reason When Making Prediction
TLDR
The experimental results show that the proposed approach can generate reasonable explanations for its predictions even with a smallscale training explanation text, which indicates that making predictions can benefit from generating the explanation in the decision process. Expand
Post-hoc Interpretability for Neural NLP: A Survey
TLDR
A categorization of how recent interpretability methods communicate explanations and discusses the methods in depth focuses on post-hoc methods, which provide explanations after a model is learned and generally model-agnostic. Expand
Grounding ‘Grounding’ in NLP
TLDR
This work investigates the gap between definitions of “grounding” in NLP and Cognitive Science, and presents ways to both create new tasks or repurpose existing ones to make advancements towards achieving a more complete sense of grounding. Expand
...
1
2
...

References

SHOWING 1-10 OF 60 REFERENCES
Language Models are Unsupervised Multitask Learners
TLDR
It is demonstrated that language models begin to learn these tasks without any explicit supervision when trained on a new dataset of millions of webpages called WebText, suggesting a promising path towards building language processing systems which learn to perform tasks from their naturally occurring demonstrations. Expand
Generating Token-Level Explanations for Natural Language Inference
TLDR
It is shown that it is possible to generate token-level explanations for NLI without the need for training data explicitly annotated for this purpose, using a simple LSTM architecture and evaluating both LIME and Anchor explanations for this task. Expand
Language Models as Knowledge Bases?
TLDR
An in-depth analysis of the relational knowledge already present (without fine-tuning) in a wide range of state-of-the-art pretrained language models finds that BERT contains relational knowledge competitive with traditional NLP methods that have some access to oracle knowledge. Expand
CommonsenseQA: A Question Answering Challenge Targeting Commonsense Knowledge
TLDR
This work presents CommonsenseQA: a challenging new dataset for commonsense question answering, which extracts from ConceptNet multiple target concepts that have the same semantic relation to a single source concept. Expand
A Simple Method for Commonsense Reasoning
TLDR
Key to this method is the use of language models, trained on a massive amount of unlabled data, to score multiple choice questions posed by commonsense reasoning tests, which outperform previous state-of-the-art methods by a large margin. Expand
QASC: A Dataset for Question Answering via Sentence Composition
TLDR
This work presents a multi-hop reasoning dataset, Question Answering via Sentence Composition (QASC), that requires retrieving facts from a large corpus and composing them to answer a multiple-choice question, and presents a two-step approach to mitigate the retrieval challenges. Expand
Explain Yourself! Leveraging Language Models for Commonsense Reasoning
TLDR
This work collects human explanations for commonsense reasoning in the form of natural language sequences and highlighted annotations in a new dataset called Common Sense Explanations to train language models to automatically generate explanations that can be used during training and inference in a novel Commonsense Auto-Generated Explanation framework. Expand
Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning
TLDR
This paper introduces Cosmos QA, a large-scale dataset of 35,600 problems that require commonsense-based reading comprehension, formulated as multiple-choice questions, and proposes a new architecture that improves over the competitive baselines. Expand
Learning Cross-Context Entity Representations from Text
Language modeling tasks, in which words, or word-pieces, are predicted on the basis of a local context, have been very effective for learning word embeddings and context dependent representations ofExpand
Annotation Artifacts in Natural Language Inference Data
TLDR
It is shown that a simple text categorization model can correctly classify the hypothesis alone in about 67% of SNLI and 53% of MultiNLI, and that specific linguistic phenomena such as negation and vagueness are highly correlated with certain inference classes. Expand
...
1
2
3
4
5
...