Obtaining Faithful Interpretations from Compositional Neural Networks

@inproceedings{Subramanian2020ObtainingFI,
  title={Obtaining Faithful Interpretations from Compositional Neural Networks},
  author={Sanjay Subramanian and Ben Bogin and Nitish Gupta and Tomer Wolfson and Sameer Singh and Jonathan Berant and Matt Gardner},
  booktitle={ACL},
  year={2020}
}
Neural module networks (NMNs) are a popular approach for modeling compositionality: they achieve high accuracy when applied to problems in language and vision, while reflecting the compositional structure of the problem in the network architecture. However, prior work implicitly assumed that the structure of the network modules, describing the abstract reasoning process, provides a faithful explanation of the model’s reasoning; that is, that all modules perform their intended behaviour. In this… Expand

Figures and Tables from this paper

Weakly Supervised Neuro-Symbolic Module Networks for Numerical Reasoning
TLDR
WNSMN is proposed, a Weakly-Supervised Neuro-Symbolic Module Network trained with answers as the sole supervision for numerical reasoning based MRC that outperforms NMN by 32% and the reasoning-free language model GenBERT by 8% in exact match accuracy when trained under comparable weak supervised settings. Expand
Explainable Multi-hop Verbal Reasoning Through Internal Monologue
TLDR
This work implements the Explainable multi-hop Verbal Reasoner (EVR) by extending the classic reasoning paradigm General Problem Solver with a SOTA generative language model to generate subgoals and perform inference in natural language at each reasoning step. Expand
Measuring and Improving Faithfulness of Attention in Neural Machine Translation
TLDR
The proposed faithfulness measure for NMT models can be improved using a novel differentiable objective that rewards faithful behaviour by the model through probability divergence and has a useful regularization effect on the NMT model and can even improve translation quality in some cases. Expand
Enriching a Model's Notion of Belief using a Persistent Memory
TLDR
This work adds a memory component a BeliefBank that records a model’s answers, and two mechanisms that use it to improve consistency among beliefs, and shows that, in a controlled experimental setting, these two mechanisms improve both accuracy and consistency. Expand
Training with Adversaries to Improve Faithfulness of Attention in Neural Machine Translation
TLDR
The proposed objective improves faithfulness without reducing the translation quality and it also seems to have a useful regularization effect on the NMT model and can even improve translation quality in some cases. Expand
A Diagnostic Study of Explainability Techniques for Text Classification
TLDR
A comprehensive list of diagnostic properties for evaluating existing explainability techniques is developed and it is found that the gradient-based explanations perform best across tasks and model architectures, and further insights into the properties are presented. Expand
Evaluating Explanations for Reading Comprehension with Realistic Counterfactuals
TLDR
This analysis suggests that pairwise explanation techniques are better suited to RC than token-level attributions, which are often unfaithful in the scenarios the authors consider, and proposes an improvement to an attention-based attribution technique, resulting in explanations which better reveal the model’s behavior. Expand
Neuro-Symbolic VQA: A review from the perspective of AGI desiderata
TLDR
This work looks at neuro-symbolic approaches to visual question answering (VQA) from the perspective of AGI desiderata, and sees how well these systems meet these desiderATA, and how the Desiderata often pull the scientist in opposing directions. Expand
A Survey on Explainability in Machine Reading Comprehension
TLDR
This paper presents a systematic review of benchmarks and approaches for explainability in Machine Reading Comprehension (MRC), and presents the evaluation methodologies to assess the performance of explainable systems. Expand
Neural Event Semantics for Grounded Language Understanding
Abstract We present a new conjunctivist framework, neural event semantics (NES), for compositional grounded language understanding. Our approach treats all words as classifiers that compose to form aExpand
...
1
2
3
...

References

SHOWING 1-10 OF 34 REFERENCES
Explainable Neural Computation via Stack Neural Module Networks
TLDR
A novel neural modular approach that performs compositional reasoning by automatically inducing a desired sub-task decomposition without relying on strong supervision is presented, which is more interpretable to human evaluators compared to other state-of-the-art models. Expand
Neural Module Networks for Reasoning over Text
TLDR
This work extends Neural module networks by introducing modules that reason over a paragraph of text, performing symbolic reasoning over numbers and dates in a probabilistic and differentiable manner, and proposing an unsupervised auxiliary loss to help extract arguments associated with the events in text. Expand
Learning to Reason: End-to-End Module Networks for Visual Question Answering
TLDR
End-to-End Module Networks are proposed, which learn to reason by directly predicting instance-specific network layouts without the aid of a parser, and achieve an error reduction of nearly 50% relative to state-of-theart attentional approaches. Expand
Learning to Compose Neural Networks for Question Answering
TLDR
A question answering model that applies to both images and structured knowledge bases that uses natural language strings to automatically assemble neural networks from a collection of composable modules that achieves state-of-the-art results on benchmark datasets. Expand
CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning
TLDR
This work presents a diagnostic dataset that tests a range of visual reasoning abilities and uses this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations. Expand
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
TLDR
A new language representation model, BERT, designed to pre-train deep bidirectional representations from unlabeled text by jointly conditioning on both left and right context in all layers, which can be fine-tuned with just one additional output layer to create state-of-the-art models for a wide range of tasks. Expand
Attention is not not Explanation
TLDR
It is shown that even when reliable adversarial distributions can be found, they don’t perform well on the simple diagnostic, indicating that prior work does not disprove the usefulness of attention mechanisms for explainability. Expand
GQA: A New Dataset for Real-World Visual Reasoning and Compositional Question Answering
We introduce GQA, a new dataset for real-world visual reasoning and compositional question answering, seeking to address key shortcomings of previous VQA datasets. We have developed a strong andExpand
Self-Assembling Modular Networks for Interpretable Multi-Hop Reasoning
TLDR
This work presents an interpretable, controller-based Self-Assembling Neural Modular Network for multi-hop reasoning, where four novel modules (Find, Relocate, Compare, NoOp) are designed to perform unique types of language reasoning. Expand
Towards Faithfully Interpretable NLP Systems: How Should We Define and Evaluate Faithfulness?
TLDR
The current binary definition for faithfulness sets a potentially unrealistic bar for being considered faithful, and is called for discarding the binary notion of faithfulness in favor of a more graded one, which is of greater practical utility. Expand
...
1
2
3
4
...