• Corpus ID: 204904776

Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks

  title={Input-Cell Attention Reduces Vanishing Saliency of Recurrent Neural Networks},
  author={Aya Abdelsalam Ismail and Mohamed K. Gunady and Luiz Pessoa and H{\'e}ctor Corrada Bravo and Soheil Feizi},
  booktitle={Neural Information Processing Systems},
Recent efforts to improve the interpretability of deep neural networks use saliency to characterize the importance of input features to predictions made by models. Work on interpretability using saliency-based methods on Recurrent Neural Networks (RNNs) has mostly targeted language tasks, and their applicability to time series data is less understood. In this work we analyze saliency-based methods for RNNs, both classical and gated cell architectures. We show that RNN saliency vanishes over… 

Figures and Tables from this paper

Benchmarking Deep Learning Interpretability in Time Series Predictions

It is shown that in general, network architectures and saliency methods fail to reliably and accurately identify feature importance over time in time series data, and the quality of saliency maps can be improved substantially by using the proposed two-step temporal saliency rescaling (TSR) approach.

Improving Deep Learning Interpretability by Saliency Guided Training

This paper introduces a saliency guided training procedure for neural networks to reduce noisy gradients used in predictions while retaining the predictive performance of the model.

Improving the Interpretability of fMRI Decoding using Deep Neural Networks and Adversarial Robustness.

A variety of methods for producing gradient-based saliency maps are reviewed, and a new adversarial training method is presented to make DNNs robust to input noise, with the goal of improving interpretability.

Evaluating Adversarial Robustness for Deep Neural Network Interpretability using fMRI Decoding

This paper develops two quantitative evaluation procedures for saliency methods, using the fact that the Human Connectome Project (HCP) dataset contains functional magnetic resonance imaging (fMRI) data from multiple tasks per subject to create ground truth saliency maps.

Core Risk Minimization using Salient ImageNet

A new learning paradigm called Core Risk Minimization (CoRM) is introduced whose objective ensures that the model predicts a class using its core features, and different computational approaches for solving CoRM achieve significantly higher core accuracy (accuracy when non-core regions corrupted using noise) with no drop in clean accuracy.

Spatiotemporal Attention for Multivariate Time Series Prediction and Interpretation

This work proposes a novel deep learning architecture, called spatiotemporal attention mechanism (STAM), for simultaneous learning of the most important time steps and variables and shows that STAM maintains state-of-the-art prediction accuracy while offering the benefit of accurate spatiotmporal interpretability.

Causal ImageNet: How to discover spurious features in Deep Learning?

A scalable framework to discover a subset of spurious and causal visual attributes used in inferences of a general model and localize them on a large number of images with minimal human supervision and shows that these neural feature annotations generalize extremely well to many more images without any human supervision.

Interpretable Sequence Classification Via Prototype Trajectory

A novel interpretable recurrent neural network (RNN) model, called ProtoryNet, is proposed, in which a new concept of prototype trajectories are introduced, which enable intuitive, fine-grained interpretation of how the model reached to the final prediction, resembling the process of how humans analyze paragraphs.

BolT: Fused Window Transformers for fMRI Time Series Analysis

BolT, a blood-oxygen-level-dependent transformer model, is presented for analyzing multi-variate fMRI time series and explanatory analyses to identify landmark time points and regions that contribute most to model decisions corroborate prominent neuroscientific findings in the literature.

EDGE: Explaining Deep Reinforcement Learning Policies

A novel self-explainable model is proposed that augments a Gaussian process with a customized kernel function and an interpretable predictor and can predict an agent’s final rewards from its game episodes and extract time step importance within episodes as strategy-level explanations for that agent.



Learning Important Features Through Propagating Activation Differences

DeepLIFT (Deep Learning Important FeaTures), a method for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input, is presented.

Attention is All you Need

A new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely is proposed, which generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.

Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization

This work proposes a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable, and shows that even non-attention based models learn to localize discriminative regions of input image.

Striving for Simplicity: The All Convolutional Net

It is found that max-pooling can simply be replaced by a convolutional layer with increased stride without loss in accuracy on several image recognition benchmarks.

Evaluating the Visualization of What a Deep Neural Network Has Learned

A general methodology based on region perturbation for evaluating ordered collections of pixels such as heatmaps and shows that the recently proposed layer-wise relevance propagation algorithm qualitatively and quantitatively provides a better explanation of what made a DNN arrive at a particular classification decision than the sensitivity-based approach or the deconvolution method.

A unified view of gradient-based attribution methods for Deep Neural Networks

This work analyzes various state-of-the-art attribution methods and proves unexplored connections between them, and performs an empirical evaluation with six attribution methods on a variety of tasks and architectures.

Show, Attend and Tell: Neural Image Caption Generation with Visual Attention

An attention based model that automatically learns to describe the content of images is introduced that can be trained in a deterministic manner using standard backpropagation techniques and stochastically by maximizing a variational lower bound.

LSTMVis: A Tool for Visual Analysis of Hidden State Dynamics in Recurrent Neural Networks

This work presents LSTMVis, a visual analysis tool for recurrent neural networks with a focus on understanding these hidden state dynamics, and describes the domain, the different stakeholders, and their goals and tasks.

Analyzing Neuroimaging Data Through Recurrent Deep Learning Models

DeepLight outperforms conventional approaches of uni- and multivariate fMRI analysis in decoding the cognitive states and in identifying the physiologically appropriate brain regions associated with these states, and is demonstrated to have the versatility to apply to a large fMRI dataset of the Human Connectome Project.