• Corpus ID: 235421605

DANCE: Enhancing saliency maps using decoys

@inproceedings{Lu2021DANCEES,
  title={DANCE: Enhancing saliency maps using decoys},
  author={Yang Young Lu and Wenbo Guo and Xinyu Xing and William Stafford Noble},
  booktitle={ICML},
  year={2021}
}

• Y. Lu, Wenbo Guo, +1 author William Stafford Noble
• Published in ICML 3 February 2020
• Computer Science

Saliency methods can make deep neural network predictions more interpretable by identifying a set of critical features in an input sample, such as pixels that contribute most strongly to a prediction made by an image classifier. Unfortunately, recent evidence suggests that many saliency methods poorly perform, especially in situations where gradients are saturated, inputs contain adversarial perturbations, or predictions rely upon interfeature dependence. To address these issues, we propose a framework, DANCE, which improves the robustness of saliency methods by following a two-step procedure. First, we introduce a perturbation mechanism that subtly varies the input sample without changing its intermediate representations. Using this approach, we can gather a corpus of perturbed (“decoy”) data samples while ensuring that the perturbed and original input samples follow similar distributions. Second, we compute saliency maps for the decoy samples and propose a new method to aggregate saliency maps. With this design, we offset influence of gradient saturation. From a theoretical perspective, we show that the aggregated saliency map not only captures inter-feature dependence but, more importantly, is robust against previously described adversarial perturbation methods. Our empirical results suggest that, both qualitatively and quantitatively, DANCE outperforms existing methods in a variety of application domains. 1 Equal contribution Department of Genome Sciences, University of Washington, Seattle, WA, USA College of Information Sciences and Technology, The Pennsylvania State University, State College, PA, USA Paul G. Allen School of Computer Science and Engineering, University of Washington, Seattle, WA, USA. Correspondence to: Xinyu Xing <[e-mail]>, William Stafford Noble <[e-mail]>. Proceedings of the 38 th International Conference on Machine Learning, PMLR 139, 2021. Copyright 2021 by the author(s). The Apache licensed source code of DANCE will be available at https://bitbucket.org/noblelab/dance.