• Corpus ID: 239024909

Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning

@inproceedings{Hamilton2021AxiomaticEF,
  title={Axiomatic Explanations for Visual Search, Retrieval, and Similarity Learning},
  author={Mark Hamilton and Scott M. Lundberg and Lei Zhang and Stephanie Fu and William T. Freeman},
  year={2021}
}
Visual search, recommendation, and contrastive similarity learning power technologies that impact billions of users worldwide. Modern model architectures can be complex and difficult to interpret, and there are several competing techniques one can use to explain a search engine’s behavior. We show that the theory of fair credit assignment provides a unique axiomatic solution that generalizes several existing recommendationand metric-explainability techniques in the literature. Using this… 
View PDF on arXiv

References

SHOWING 1-10 OF 74 REFERENCES
Explaining Explanations: Axiomatic Feature Interactions for Deep Networks
TLDR
This work presents Integrated Hessians, an extension of Integrated Gradients that explains pairwise feature interactions in neural networks and finds that the method is faster than existing methods when the number of features is large, and outperforms previous methods on existing quantitative benchmarks.
Visual Explanation for Deep Metric Learning
TLDR
This work proposes an intuitive idea to show where contributes the most to the overall similarity of two input images by decomposing the final activation by generating point-to-point activation intensity between two images so that the relationship between different regions is uncovered.
Axiomatic Attribution for Deep Networks
We study the problem of attributing the prediction of a deep network to its input features, a problem previously studied by several other works. We identify two fundamental axioms— Sensitivity and…
Very Deep Convolutional Networks for Large-Scale Image Recognition
TLDR
This work investigates the effect of the convolutional network depth on its accuracy in the large-scale image recognition setting using an architecture with very small convolution filters, which shows that a significant improvement on the prior-art configurations can be achieved by pushing the depth to 16-19 weight layers.
A Unified Approach to Interpreting Model Predictions
TLDR
A unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations), which unifies six existing methods and presents new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.
PyTorch: An Imperative Style, High-Performance Deep Learning Library
TLDR
This paper details the principles that drove the implementation of PyTorch and how they are reflected in its architecture, and explains how the careful and pragmatic implementation of the key components of its runtime enables them to work together to achieve compelling performance.
A Simplified Bargaining Model for the n-Person Cooperative Game
The bargaining model I propose to discuss in this paper2 is a simplified and, as I hope, improved version of my bargaining model published in the Princeton Contributions (see [3]). Like my earlier…
The Shapley Taylor Interaction Index
TLDR
A generalization of the Shapley value called Shapley-Taylor index is proposed that attributes the model's prediction to interactions of subsets of features up to some size k that is analogous to how the truncated Taylor Series decomposes the function value at a certain point using its derivatives at a different point.
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
TLDR
This work proposes a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable, and shows that even non-attention based models learn to localize discriminative regions of input image.
Deep Residual Learning for Image Recognition
TLDR
This work presents a residual learning framework to ease the training of networks that are substantially deeper than those used previously, and provides comprehensive empirical evidence showing that these residual networks are easier to optimize, and can gain accuracy from considerably increased depth.
...
...