Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy
@article{Slack2021DefuseHU, title={Defuse: Harnessing Unrestricted Adversarial Examples for Debugging Models Beyond Test Accuracy}, author={Dylan Slack and N. Rauschmayr and K. Kenthapadi}, journal={ArXiv}, year={2021}, volume={abs/2102.06162} }
We typically compute aggregate statistics on held-out test data to assess the generalization of machine learning models. However, statistics on test data often overstate model generalization, and thus, the performance of deployed machine learning models can be variable and untrustworthy. Motivated by these concerns, we develop methods to automatically discover and correct model errors beyond those available in the data. We propose Defuse, a method that generates novel model misclassifications… Expand
Figures from this paper
figure 1 figure 2 figure 3 figure 4 figure 5 figure 6 figure 7 figure 8 figure 9 figure 10 figure 11 figure 12 figure 13 figure 14 figure 16 figure 17 figure 18 figure 19 figure 20 figure 21 figure 22 figure 23 figure 24 figure 25 figure 26 figure 27 figure 28 figure 29 figure 30 figure 31 figure 32 figure 33 figure 34 figure 35 figure 36 figure 37 figure 38 figure 39 figure 40 figure 41
References
SHOWING 1-10 OF 39 REFERENCES
Generating Natural Adversarial Examples
- Computer Science, Mathematics
- ICLR
- 2018
- 275
- Highly Influential
- PDF
TensorFuzz: Debugging Neural Networks with Coverage-Guided Fuzzing
- Computer Science, Mathematics
- ICML
- 2019
- 123
- PDF
Constructing Unrestricted Adversarial Examples with Generative Models
- Computer Science, Mathematics
- NeurIPS
- 2018
- 120
- Highly Influential
- PDF
Socratic Learning: Augmenting Generative Models to Incorporate Latent Subsets in Training Data
- Computer Science, Mathematics
- 2016
- 21
- PDF