Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models

  title={Concept Saliency Maps to Visualize Relevant Features in Deep Generative Models},
  author={Lennart Brocki and Neo Christopher Chung},
  journal={2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA)},
  • L. BrockiN. C. Chung
  • Published 29 October 2019
  • Computer Science, Environmental Science
  • 2019 18th IEEE International Conference On Machine Learning And Applications (ICMLA)
Evaluating, explaining, and visualizing high-level concepts in generative models, such as variational autoencoders (VAEs), is challenging in part due to a lack of known prediction classes that are required to generate saliency maps in supervised learning. While saliency maps may help identify relevant features (e.g., pixels) in the input for classification tasks of deep neural networks, similar frameworks are understudied in unsupervised learning. Therefore, we introduce a new method of… 

Figures and Tables from this paper

Input Bias in Rectified Gradients and Modified Saliency Maps

  • L. BrockiN. C. Chung
  • Environmental Science
    2021 IEEE International Conference on Big Data and Smart Computing (BigComp)
  • 2021
It is demonstrated that dark areas of an input image are not highlighted by a saliency map using Rectified Gradients, even if it is relevant for the class or concept, which showcases how a visual criteria may not align with true explainability of deep learning models.

Removing Brightness Bias in Rectified Gradients

It is demonstrated that dark areas of an input image are not highlighted by a saliency map using Rectified Gradients, even if it is relevant for the class or concept, and a brightness bias is identified.

Concept Activation Regions: A Generalized Framework For Concept-Based Explanations

This work introduces an extension of the CAV formalism that is based on the kernel trick and support vector classifiers that yields global concept-based explanations and local conceptbased feature importance, and proves that CAR explanations built with radial kernels are invariant under latent space isometries.

Label-Free Explainability for Unsupervised Models

This work introduces two crucial extensions of post-hoc explanation techniques that respectively highlight influential features and training examples for a black-box to construct representations at inference time and demonstrates that these extensions can be successfully implemented as simple wrap-pers around many existing feature and example importance methods.

"Teaching Independent Parts Separately" (TIPSy-GAN) : Improving Accuracy and Stability in Unsupervised Adversarial 2D to 3D Pose Estimation

It is shown that for two independent generators training adversarially has improved stability than that of a solo generator which collapses, and that improving the self-consistency cycle is key to lowering the evaluation error and therefore introduce new consistency constraints during training.

Interpretable Autoencoders Trained on Single Cell Sequencing Data Can Transfer Directly to Data from Unseen Tissues

It is shown that carefully trained autoencoders can be used to assist the interpretation of new unseen data, and can generalize to the extent that they can transfer directly without additional training.

Debiasing Concept Bottleneck Models with Instrumental Variables

The problem of the concepts being correlated with confounding information in the features is studied and a new causal prior graph for modeling the impacts of unobserved variables and a method to remove the impact of confounding information and noise using the instrumental variable techniques are proposed.

Debiasing Concept-based Explanations with Causal Analysis

This work proposes a new causal prior graph for modeling the impacts of unobserved variables and a method to remove the impact of confounding information and noise using a two-stage regression technique borrowed from the instrumental variable literature.

Application of artificial intelligence in nuclear medicine and molecular imaging: a review of current status and future perspectives for clinical translation

The potential applications of AI in the field are focused on, both from a physical (radiomics, underlying statistics, image reconstruction and data analysis) and a clinical (neurology, cardiology, oncology) perspective.

Explaining, Evaluating and Enhancing Neural Networks' Learned Representations

This work proposes a natural aggregation method generalizing attribution maps between any two (convolutional) layers of a neural network and employs such attributions to evaluate the informativeness and the disentanglement of latent embeddings and shows that adopting the proposed scores as constraints during the training of a representation learning task improves the downstream performance of the model.



Sanity Checks for Saliency Maps

It is shown that some existing saliency methods are independent both of the model and of the data generating process, and methods that fail the proposed tests are inadequate for tasks that are sensitive to either data or model.

Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps

This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets), and establishes the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks.

Why are Saliency Maps Noisy? Cause of and Solution to Noisy Saliency Maps

A new hypothesis that noise may occur in saliency maps when irrelevant features pass through ReLU activation functions is proposed and Rectified Gradient, a method that alleviates this problem through layer-wise thresholding during backpropagation is proposed.

Unsupervised Learning of Discriminative Attributes and Visual Representations

This work first train a CNN coupled with unsupervised discriminative clustering, and then uses the cluster membership as a soft supervision to discover shared attributes from the clusters while maximizing their separability.

beta-VAE: Learning Basic Visual Concepts with a Constrained Variational Framework

Learning an interpretable factorised representation of the independent data generative factors of the world without supervision is an important precursor for the development of artificial

Interpretability Beyond Feature Attribution: Quantitative Testing with Concept Activation Vectors (TCAV)

Concept Activation Vectors (CAVs) are introduced, which provide an interpretation of a neural net's internal state in terms of human-friendly concepts, and may be used to explore hypotheses and generate insights for a standard image classification network as well as a medical application.

Visualizing Higher-Layer Features of a Deep Network

This paper contrast and compare several techniques applied on Stacked Denoising Autoencoders and Deep Belief Networks, trained on several vision datasets, and shows that good qualitative interpretations of high level features represented by such models are possible at the unit level.

Evaluating the Visualization of What a Deep Neural Network Has Learned

A general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps and shows that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method.

Learning Important Features Through Propagating Activation Differences

DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input, is presented.

Autoencoding beyond pixels using a learned similarity metric

An autoencoder that leverages learned representations to better measure similarities in data space is presented and it is shown that the method learns an embedding in which high-level abstract visual features (e.g. wearing glasses) can be modified using simple arithmetic.