Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging

@article{Arun2020AssessingT,
  title={Assessing the (Un)Trustworthiness of Saliency Maps for Localizing Abnormalities in Medical Imaging},
  author={Nishanth Thumbavanam Arun and N. Gaw and P. Singh and K. Chang and Mehak Aggarwal and B. Chen and Katherine Hoebel and S. Gupta and Jay B. Patel and Mishka Gidwani and Julius Adebayo and M D Li and Jayashree Kalpathy-Cramer},
  journal={ArXiv},
  year={2020},
  volume={abs/2008.02766}
}
Saliency maps have become a widely used method to make deep learning models more interpretable by providing post-hoc explanations of classifiers through identification of the most pertinent areas of the input medical image. They are increasingly being used in medical imaging to provide clinically plausible explanations for the decisions the neural network makes. However, the utility and robustness of these visualization maps has not yet been rigorously examined in the context of medical imaging… Expand

Topics from this paper

Deep learning saliency maps do not accurately highlight diagnostically relevant regions for medical image interpretation
TLDR
It is demonstrated that the most commonly used saliency map generating method, Grad-CAM, results in low performance for 10 pathologies on chest X-rays and demonstrates that several important limitations of interpretability techniques for medical imaging must be addressed before use in clinical workflows. Expand
Clinical Validation of Saliency Maps for Understanding Deep Neural Networks in Ophthalmology
TLDR
This study used two different network architectures and developed ensembles of DNNs to detect diabetic retinopathy and neovascular age-related macular degeneration from retinal fundus images and optical coherence tomography scans, respectively, and systematically validated saliency maps against clinicians. Expand
Towards Trainable Saliency Maps in Medical Imaging
TLDR
This study validate a model design element agnostic to both architecture complexity and model task, and show how introducing this element gives an inherently self-explanatory model. Expand
On the Overlap Between Grad-CAM Saliency Maps and Explainable Visual Features in Skin Cancer Images
TLDR
It is found that the best threshold value, i.e., the threshold at which one can measure the highest Jaccard index, varies significantly among features; ranging from 0.3 to 0.7, shows that higher resolution saliency maps could help doctors in spotting wrong classifications. Expand
CheXseg: Combining Expert Annotations with DNN-generated Saliency Maps for X-ray Segmentation
TLDR
This work proposes a method that combines the high quality of pixel-level expert annotations with the scale of coarse DNN-generated saliency maps for training multi-label semantic segmentation models and implements a semi-supervised method using knowledge distillation. Expand
When saliency goes off on a tangent: Interpreting Deep Neural Networks with nonlinear saliency maps
TLDR
A novel class of methods for generating nonlinear saliency maps which fully account for the nonlinearity of the underlying learning system and significantly aids interpretability of deep neural networks and related machine learning systems. Expand
A Review of Explainable Deep Learning Cancer Detection Models in Medical Imaging
TLDR
A review of the deep learning explanation literature focused on cancer detection using MR images is presented and the gap between what clinicians deem explainable and what current methods provide is discussed and future suggestions to close this gap are provided. Expand
Interpretable Mammographic Image Classification using Cased-Based Reasoning and Deep Learning
TLDR
A novel interpretable neural network algorithm that uses casebased reasoning for mammography to aid a radiologist in their decisions, which detects clinical features with equal or higher accuracy, provides a more detailed explanation of its prediction, and is better able to differentiate the classification-relevant parts of the image. Expand
Optimising Knee Injury Detection with Spatial Attention and Validating Localisation Ability
TLDR
This work employs a pre-trained, multi-view Convolutional Neural Network (CNN) with a spatial attention block to optimise knee injury detection and developed an objective metric, Penalised Localisation Accuracy (PLA), to validate the model’s localisation ability. Expand
Applications of interpretability in deep learning models for ophthalmology
TLDR
The concept of model interpretability is introduced, its applications in deep learning models for clinical ophthalmology are reviewed, and its role in the integration of artificial intelligence in healthcare is discussed. Expand
...
1
2
3
4
...

References

SHOWING 1-10 OF 76 REFERENCES
Assessing the validity of saliency maps for abnormality localization in medical imaging
TLDR
This work explored the credibility of the various existing saliency map methods on the RSNA Pneumonia dataset and found that GradCAM was the most sensitive to model parameter and label randomization, and was highly agnostic to model architecture. Expand
Sanity Checks for Saliency Maps
TLDR
It is shown that some existing saliency methods are independent both of the model and of the data generating process, and methods that fail the proposed tests are inadequate for tasks that are sensitive to either data or model. Expand
XRAI: Better Attributions Through Regions
TLDR
A novel region-based attribution method that builds upon integrated gradients and contributes an axiom-based sanity check for attribution methods is presented and it is shown that XRAI produces better results than other saliency methods for common models and the ImageNet dataset. Expand
Deep Inside Convolutional Networks: Visualising Image Classification Models and Saliency Maps
TLDR
This paper addresses the visualisation of image classification models, learnt using deep Convolutional Networks (ConvNets), and establishes the connection between the gradient-based ConvNet visualisation methods and deconvolutional networks. Expand
Testing the robustness of attribution methods for convolutional neural networks in MRI-based Alzheimer's disease classification
TLDR
It is shown that visual comparison is not sufficient and that some widely used attribution methods produce highly inconsistent outcomes, including gradient*input, guided backpropagation, layer-wise relevance propagation and occlusion. Expand
Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization
TLDR
This work proposes a technique for producing ‘visual explanations’ for decisions from a large class of Convolutional Neural Network (CNN)-based models, making them more transparent and explainable, and shows that even non-attention based models learn to localize discriminative regions of input image. Expand
Multi-Institutional Assessment and Crowdsourcing Evaluation of Deep Learning for Automated Classification of Breast Density.
TLDR
Deep learning algorithms to automatically assess BI-RADS breast density were developed and crowdsourced annotations, including those from attendees who routinely read mammograms, had higher agreement with the algorithm than with the original interpreting radiologists. Expand
Learning Deep Features for Discriminative Localization
In this work, we revisit the global average pooling layer proposed in [13], and shed light on how it explicitly enables the convolutional neural network (CNN) to have remarkable localization abilityExpand
Deep neural network or dermatologist?
TLDR
It is shown that despite high accuracy, the models will occasionally assign importance to features that are not relevant to the diagnostic task, representing first steps in bridging the gap between model accuracy and interpretability in the domain of skin cancer classification. Expand
Siamese neural networks for continuous disease severity evaluation and change detection in medical imaging
TLDR
A convolutional Siamese neural network approach to evaluate disease severity at single time points and change between longitudinal patient visits on a continuous spectrum of disease severity and change in medical imaging is developed. Expand
...
1
2
3
4
5
...