• Corpus ID: 212725475

Towards Ground Truth Evaluation of Visual Explanations

  title={Towards Ground Truth Evaluation of Visual Explanations},
  author={Ahmed Osman and Leila Arras and Wojciech Samek},
Several methods have been proposed to explain the decisions of neural networks in the visual domain via saliency heatmaps (aka relevances/feature importance scores). Thus far, these methods were mainly validated on real-world images, using either pixel perturbation experiments or bounding box localization accuracies. In the present work, we propose instead to evaluate explanations in a restricted and controlled setup using a synthetic dataset of rendered 3D shapes. To this end, we generate a… 

Quantitative Evaluation of Machine Learning Explanations: A Human-Grounded Benchmark

A benchmark for image and text domains using multi-layer human attention masks aggregated from multiple human annotators is proposed and its utility for quantitative evaluation of model explanations is demonstrated by comparing it with human subjective ratings and ground-truth single-layer segmentation masks evaluations.

Found a Reason for me? Weakly-supervised Grounded Visual Question Answering using Capsules

This paper proposes a visual capsule module with a query-based selection mechanism of capsule features, that allows the model to focus on relevant regions based on the textual cues about visual information in the question and shows that integrating the proposed capsule module in existing VQA systems significantly improves their performance on the weakly supervised grounding task.

Towards Measuring Bias in Image Classification

This work presents a systematic approach to uncover data bias by means of attribution maps and shows that some attribution map techniques highlight the presence of bias in the data better than others and metrics can support the identification of bias.

Evaluating Attribution for Graph Neural Networks

This work adapt commonly-used attribution methods for GNNs and quantitatively evaluate them using the axes of attribution accuracy, stability, faithfulness and consistency, and makes concrete recommendations for which attribution methods to use.

Learning to Explain: Generating Stable Explanations Fast

This paper proposes a Learning to Explain approach that learns the behaviour of an underlying explanation algorithm simultaneously from all training examples, and once the explanation algorithm is distilled into an explainer network, it can be used to explain new instances.

Deep Interpretable Classification and Weakly-Supervised Segmentation of Histology Images via Max-Min Uncertainty

High uncertainty is introduced as a criterion to localize non-discriminative regions that do not affect classifier decision, and is described with original Kullback-Leibler (KL) divergence losses evaluating the deviation of posterior predictions from the uniform distribution.

On the Robustness of Pretraining and Self-Supervision for a Deep Learning-based Analysis of Diabetic Retinopathy

The results indicate that models initialized from ImageNet pretraining report a significant increase in performance, generalization and robustness to image distortions, and self-supervised models show further benefits to supervised models.

Explaining the Decisions of Convolutional and Recurrent Neural Networks

This chapter describes Layer-wise Relevance Propagation (LRP), a propagation-based explanation technique that can explain the decisions of a variety of ML models, including state-of-the-art convolutional and recurrent neural networks.

Vulnerabilities of Connectionist AI Applications: Evaluation and Defense

It is concluded that single protective measures are not sufficient but rather multiple measures on different levels have to be combined to achieve a minimum level of IT security for AI applications.

Evaluating and Aggregating Feature-based Model Explanations

This paper develops a procedure for learning an aggregate explanation function with lower complexity and then derive a new aggregate Shapley value explanation function that minimizes sensitivity.



Sanity Checks for Saliency Maps

It is shown that some existing saliency methods are independent both of the model and of the data generating process, and methods that fail the proposed tests are inadequate for tasks that are sensitive to either data or model.

Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks

Evaluation on the MNIST, ILSVRC 12 and Fashion 144k datasets quantitatively shows that the proposed method is able to identify relevant internal features for the classes of interest while improving the quality of the produced visualizations.

Evaluating the Visualization of What a Deep Neural Network Has Learned

A general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps and shows that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method.

Top-Down Neural Attention by Excitation Backprop

A new backpropagation scheme, called Excitation Backprop, is proposed to pass along top-down signals downwards in the network hierarchy via a probabilistic Winner-Take-All process, and the concept of contrastive attention is introduced to make the top- down attention maps more discriminative.

Visualizing and Understanding Convolutional Networks

A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark.

On Pixel-Wise Explanations for Non-Linear Classifier Decisions by Layer-Wise Relevance Propagation

This work proposes a general solution to the problem of understanding classification decisions by pixel-wise decomposition of nonlinear classifiers by introducing a methodology that allows to visualize the contributions of single pixels to predictions for kernel-based classifiers over Bag of Words features and for multilayered neural networks.

Towards better understanding of gradient-based attribution methods for Deep Neural Networks

This work analyzes four gradient-based attribution methods and formally prove conditions of equivalence and approximation between them, and constructs a unified framework which enables a direct comparison, as well as an easier implementation.

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets), and establishes the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks.

Layer-Wise Relevance Propagation: An Overview

This chapter gives a concise introduction to LRP with a discussion of how to implement propagation rules easily and efficiently, how the propagation procedure can be theoretically justified as a ‘deep Taylor decomposition’, how to choose the propagation rules at each layer to deliver high explanation quality, and how LRP can be extended to handle a variety of machine learning scenarios beyond deep neural networks.

CLEVR: A Diagnostic Dataset for Compositional Language and Elementary Visual Reasoning

This work presents a diagnostic dataset that tests a range of visual reasoning abilities and uses this dataset to analyze a variety of modern visual reasoning systems, providing novel insights into their abilities and limitations.