Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks

@article{Fong2018Net2VecQA,
  title={Net2Vec: Quantifying and Explaining How Concepts are Encoded by Filters in Deep Neural Networks},
  author={Ruth Fong and Andrea Vedaldi},
  journal={2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition},
  year={2018},
  pages={8730-8738}
}
  • Ruth Fong, A. Vedaldi
  • Published 2018
  • Computer Science, Mathematics
  • 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition
In an effort to understand the meaning of the intermediate representations captured by deep networks, recent papers have tried to associate specific semantic concepts to individual neural network filter responses, where interesting correlations are often found, largely by focusing on extremal filter responses. [...] Key Method In order to investigate this idea while enabling systematic visualization and quantification of multiple filter responses, we introduce the Net2Vec framework, in which semantic concepts…Expand
Making Sense of CNNs: Interpreting Deep Representations & Their Invariances with INNs
TLDR
This work presents an approach based on INNs that recovers the task-specific, learned invariances by disentangling the remaining factor of variation in the data and invertibly transforms these recoveredinvariances combined with the model representation into an equally expressive one with accessible semantic concepts. Expand
Summit: Scaling Deep Learning Interpretability by Visualizing Activation and Attribution Summarizations
Deep learning is increasingly used in decision-making tasks. However, understanding how neural networks produce final predictions remains a fundamental challenge. Existing work on interpreting neuralExpand
A Disentangling Invertible Interpretation Network for Explaining Latent Representations
TLDR
The proposed invertible interpretation network can be transparently applied on top of existing architectures with no need to modify or retrain them and presents an efficient approach to define semantic concepts by only sketching two images and also an unsupervised strategy. Expand
Quantifying Learnability and Describability of Visual Concepts Emerging in Representation Learning
TLDR
This paper considers in particular how to characterise visual groupings discovered automatically by deep neural networks, starting with state-of-the-art clustering methods, and proposes a class-level captioning system to generate descriptions forVisual groupings automatically and compare it to human annotators using the describability metric. Expand
Understanding Deep Networks via Extremal Perturbations and Smooth Masks
TLDR
Some of the shortcomings of existing approaches to perturbation analysis are discussed and the concept of extremal perturbations are introduced, which are theoretically grounded and interpretable and allow us to remove all tunable weighing factors from the optimization problem. Expand
Contextual Semantic Interpretability
TLDR
A two-layer semantic bottleneck that gathers attributes into interpretable, sparse groups, allowing them contribute differently to the final output depending on the context, and training the semantic interpretable bottleneck using an auxiliary database. Expand
Towards Global Explanations of Convolutional Neural Networks With Concept Attribution
TLDR
AfI first conducts a feature occlusion analysis, which resembles a process of attacking models to derive the category-wide importance of different features, and then maps the feature importance to concept importance through ad-hoc semantic tasks. Expand
Visual Explanation by Interpretation: Improving Visual Feedback Capabilities of Deep Neural Networks
TLDR
Evaluation on the MNIST, ILSVRC 12 and Fashion 144k datasets quantitatively shows that the proposed method is able to identify relevant internal features for the classes of interest while improving the quality of the produced visualizations. Expand
NeuroCartography: Scalable Automatic Visual Summarization of Concepts in Deep Neural Networks
  • Haekyu Park, Nilaksh Das, +4 authors Duen Horng Chau
  • Medicine, Computer Science
  • IEEE transactions on visualization and computer graphics
  • 2021
TLDR
An interactive system that scalably summarizes and visualizes concepts learned by neural networks, which automatically discovers and groups neurons that detect the same concepts, and describes how such neuron groups interact to form higher-level concepts and the subsequent predictions. Expand
DEEP NEURAL NETWORKS
Interpretation and explanation of deep models is critical towards wide adoption of systems that rely on them. In this paper, we propose a novel scheme for both interpretation as well as explanationExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 26 REFERENCES
Network Dissection: Quantifying Interpretability of Deep Visual Representations
TLDR
This work uses the proposed Network Dissection method to test the hypothesis that interpretability is an axis-independent property of the representation space, then applies the method to compare the latent representations of various networks when trained to solve different classification problems. Expand
Do Semantic Parts Emerge in Convolutional Neural Networks?
TLDR
This work investigates the responses of convolutional filters and tries to associate their stimuli with semantic parts, and determines which semantic parts are the most discriminative and whether they correspond to those parts emerging in the network. Expand
Unsupervised learning of object semantic parts from internal states of CNNs by population encoding
TLDR
This work provides a new unsupervised method to learn semantic parts and gives new understanding of the internal representations of CNNs, based on the hypothesis that semantic parts are represented by populations of neurons rather than by single filters. Expand
Intriguing properties of neural networks
TLDR
It is found that there is no distinction between individual highlevel units and random linear combinations of high level units, according to various methods of unit analysis, and it is suggested that it is the space, rather than the individual units, that contains of the semantic information in the high layers of neural networks. Expand
Unifying Visual-Semantic Embeddings with Multimodal Neural Language Models
TLDR
This work introduces the structure-content neural language model that disentangles the structure of a sentence to its content, conditioned on representations produced by the encoder, and shows that with linear encoders, the learned embedding space captures multimodal regularities in terms of vector space arithmetic. Expand
Visualizing and Understanding Convolutional Networks
TLDR
A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. Expand
Understanding deep image representations by inverting them
Image representations, from SIFT and Bag of Visual Words to Convolutional Neural Networks (CNNs), are a crucial component of almost any image understanding system. Nevertheless, our understanding ofExpand
Synthesizing the preferred inputs for neurons in neural networks via deep generator networks
TLDR
This work dramatically improves the qualitative state of the art of activation maximization by harnessing a powerful, learned prior: a deep generator network (DGN), which generates qualitatively state-of-the-art synthetic images that look almost real. Expand
Understanding deep learning requires rethinking generalization
TLDR
These experiments establish that state-of-the-art convolutional networks for image classification trained with stochastic gradient methods easily fit a random labeling of the training data, and confirm that simple depth two neural networks already have perfect finite sample expressivity. Expand
Building high-level features using large scale unsupervised learning
TLDR
Contrary to what appears to be a widely-held intuition, the experimental results reveal that it is possible to train a face detector without having to label images as containing a face or not. Expand
...
1
2
3
...