Corpus ID: 237532222

Explainability Requires Interactivity

@article{Kirchler2021ExplainabilityRI,
  title={Explainability Requires Interactivity},
  author={Matthias Kirchler and M. Graf and M. Kloft and C. Lippert},
  journal={ArXiv},
  year={2021},
  volume={abs/2109.07869}
}
When explaining the decisions of deep neural networks, simple stories are tempting but dangerous. Especially in computer vision, the most popular explanation approaches give a false sense of comprehension to its users and provide an overly simplistic picture. We introduce an interactive framework to understand the highly complex decision boundaries of modern vision models. It allows the user to exhaustively inspect, probe, and test a network’s decisions. Across a range of case studies, we… Expand

Figures and Tables from this paper

References

SHOWING 1-10 OF 57 REFERENCES
Interpretable Explanations of Black Boxes by Meaningful Perturbation
  • Ruth C. Fong, A. Vedaldi
  • Computer Science, Mathematics
  • 2017 IEEE International Conference on Computer Vision (ICCV)
  • 2017
TLDR
A general framework for learning different kinds of explanations for any black box algorithm is proposed and the framework to find the part of an image most responsible for a classifier decision is specialised. Expand
Sanity Checks for Saliency Maps
TLDR
It is shown that some existing saliency methods are independent both of the model and of the data generating process, and methods that fail the proposed tests are inadequate for tasks that are sensitive to either data or model. Expand
This looks like that: deep learning for interpretable image recognition
TLDR
A deep network architecture -- prototypical part network (ProtoPNet), that reasons in a similar way to the way ornithologists, physicians, and others would explain to people on how to solve challenging image classification tasks, that provides a level of interpretability that is absent in other interpretable deep models. Expand
Interactive Classification for Deep Learning Interpretation
TLDR
Using modern web technologies to run in-browser inference, users can remove image features using inpainting algorithms and obtain new classifications in real time, which allows them to ask a variety of "what if" questions by experimentally modifying images and seeing how the model reacts. Expand
Explanation by Progressive Exaggeration
TLDR
This work proposes a method that explains the outcome of a classification black-box by gradually exaggerating the semantic effect of a given class by gradually changes the posterior probability from its original class to its negation. Expand
Intriguing properties of neural networks
TLDR
It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. Expand
Axiomatic Attribution for Deep Networks
We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms— Sensitivity andExpand
Learning Important Features Through Propagating Activation Differences
TLDR
DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input, is presented. Expand
Unmasking Clever Hans predictors and assessing what machines really learn
TLDR
The authors investigate how these methods approach learning in order to assess the dependability of their decision making and propose a semi-automated Spectral Relevance Analysis that provides a practically effective way of characterizing and validating the behavior of nonlinear learning machines. Expand
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
TLDR
This work proposes a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable, and shows that even non-attention based models learn to localize discriminative regions of input image. Expand
...
1
2
3
4
5
...