Corpus ID: 15127402

Visualizing Higher-Layer Features of a Deep Network

@inproceedings{Erhan2009VisualizingHF,
  title={Visualizing Higher-Layer Features of a Deep Network},
  author={D. Erhan and Yoshua Bengio and Aaron C. Courville and Pascal Vincent},
  year={2009}
}
Deep architectures have demonstrated state-of-the-art results in a variety of settings, especially with vision datasets. Beyond the model definitions and the quantitative analyses, there is a need for qualitative comparisons of the solutions learned by various deep architectures. The goal of this paper is to find good qualitative interpretations of high level features represented by such models. To this end, we contrast and compare several techniques applied on Stacked Denoising Autoencoders… Expand
igva2012.wikispaces.asu.edu
909 Citations
Highly Influential Citations
71
Background Citations
379
Methods Citations
346
Results Citations
12

909 Citations

Citation Type

Citation Type

Visualizing stacked autoencoder language learning
TLDR
A method for visualizing a deep autoencoder’s hidden layers when trained on natural language data is presented and it can help determine an appropriate stopping point in the training process and provide researchers insight into the semantic language features the network has extracted from the dataset. Expand
Visualizing and Understanding Convolutional Networks
TLDR
A novel visualization technique is introduced that gives insight into the function of intermediate feature layers and the operation of the classifier in large Convolutional Network models, used in a diagnostic role to find model architectures that outperform Krizhevsky et al on the ImageNet classification benchmark. Expand
Generative adversarial network for visualizing convolutional network
TLDR
A new visualization framework based on generative adversarial network (GAN) is introduced to give an insight into the neuron activities and what the CNN has learned to help to understand how the CNN works internally. Expand
Using effective dimension to analyze feature transformations in deep neural networks
TLDR
This paper uses singular value decomposition to analyze the relevant variations in feature space and shows that across datasets and architectures, the effective dimension of a class increases before decreasing further into the network, suggesting an initial whitening transformation. Expand
A CHAIN GRAPH INTERPRETATION
  • 2020
The last decade has witnessed a boom of deep learning research and applications achieving state-of-the-art results in various domains. However, most advances have been established empirically, and… Expand
Modeling Latent Attention Within Neural Networks
TLDR
This work presents a general method for visualizing an arbitrary neural network’s inner mechanisms and their power and limitations, and produces visualizations of how a trained network attends to components of its inputs. Expand
Towards Better Analysis of Deep Convolutional Neural Networks
TLDR
A hybrid visualization is developed to disclose the multiple facets of each neuron and the interactions between them and a biclustering-based edge bundling method is proposed to reduce visual clutter caused by a large number of connections between neurons. Expand
Interpretability of deep learning models: A survey of results
  • Supriyo Chakraborty, Richard J. Tomsett, +12 authors Prudhvi K. Gurram
  • Computer Science
  • 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI)
  • 2017
TLDR
Some of the dimensions that are useful for model interpretability are outlined, and prior work along those dimensions are categorized, in the process of performing a gap analysis of what needs to be done to improve modelinterpretability. Expand
Exploring LOTS in Deep Neural Networks
TLDR
The layerwise origin-target synthesis (LOTS) is introduced that can serve multiple purposes and can be used as a visualization technique that gives insights into the function of any intermediate feature layer by showing the notion of a particular input in deep neural networks. Expand
These do not Look Like Those: An Interpretable Deep Learning Model for Image Recognition
TLDR
A deep learning model: negative-positive prototypical part network (NP-ProtoPNet) that attempts to imitate human reasoning for image recognition while comparing the parts of a test image with the corresponding parts of the images from known classes. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 20 REFERENCES
Learning Deep Architectures for AI
TLDR
The motivations and principles regarding learning algorithms for deep architectures, in particular those exploiting as building blocks unsupervised learning of single-layer modelssuch as Restricted Boltzmann Machines, used to construct deeper models such as Deep Belief Networks are discussed. Expand
Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations
TLDR
The convolutional deep belief network is presented, a hierarchical generative model which scales to realistic image sizes and is translation-invariant and supports efficient bottom-up and top-down probabilistic inference. Expand
Greedy Layer-Wise Training of Deep Networks
TLDR
These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy mostly helps the optimization, by initializing weights in a region near a good local minimum, giving rise to internal distributed representations that are high-level abstractions of the input, bringing better generalization. Expand
An empirical evaluation of deep architectures on problems with many factors of variation
TLDR
A series of experiments indicate that these models with deep architectures show promise in solving harder learning problems that exhibit many factors of variation. Expand
Extracting and composing robust features with denoising autoencoders
TLDR
This work introduces and motivate a new training principle for unsupervised learning of a representation based on the idea of making the learned representations robust to partial corruption of the input pattern. Expand
Sparse deep belief net model for visual area V2
TLDR
An unsupervised learning model is presented that faithfully mimics certain properties of visual area V2 and the encoding of these more complex "corner" features matches well with the results from the Ito & Komatsu's study of biological V2 responses, suggesting that this sparse variant of deep belief networks holds promise for modeling more higher-order features. Expand
Exploring Strategies for Training Deep Neural Networks
TLDR
These experiments confirm the hypothesis that the greedy layer-wise unsupervised training strategy helps the optimization by initializing weights in a region near a good local minimum, but also implicitly acts as a sort of regularization that brings better generalization and encourages internal distributed representations that are high-level abstractions of the input. Expand
Sparse Feature Learning for Deep Belief Networks
TLDR
This work proposes a simple criterion to compare and select different unsupervised machines based on the trade-off between the reconstruction error and the information content of the representation, and describes a novel and efficient algorithm to learn sparse representations. Expand
Deep learning via semi-supervised embedding
We show how nonlinear embedding algorithms popular for use with shallow semi-supervised learning techniques such as kernel methods can be applied to deep multilayer architectures, either as a… Expand
Modeling image patches with a directed hierarchy of Markov random fields
TLDR
An efficient learning procedure for multilayer generative models that combine the best aspects of Markov random fields and deep, directed belief nets is described and it is shown that this type of model is good at capturing the statistics of patches of natural images. Expand
...
1
2
...