Benchmarking Neural Network Robustness to Common Corruptions and Perturbations
@article{Hendrycks2019BenchmarkingNN, title={Benchmarking Neural Network Robustness to Common Corruptions and Perturbations}, author={Dan Hendrycks and Thomas G. Dietterich}, journal={ArXiv}, year={2019}, volume={abs/1903.12261} }
In this paper we establish rigorous benchmarks for image classifier robustness. [] Key Result Together our benchmarks may aid future work toward networks that robustly generalize.
Figures and Tables from this paper
1,141 Citations
Improving Robustness of DNNs against Common Corruptions via Gaussian Adversarial Training
- Computer Science2020 IEEE International Conference on Visual Communications and Image Processing (VCIP)
- 2020
This work proposes to train neural networks with adversarial examples where the perturbations are Gaussian-distributed, and shows that the proposed GAT can improve neural networks’ robustness to noise corruptions more than other baseline methods.
Are Adversarial Robustness and Common Perturbation Robustness Independent Attributes ?
- Computer Science2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW)
- 2019
It is shown that increasing the robustness to carefully selected common perturbations, can make neural networks more robust to unseen common perturgations and it is proved that adversarial robustness and robustness of neural networks to common perturations are independent.
Improving Corruption and Adversarial Robustness by Enhancing Weak Subnets
- Computer ScienceArXiv
- 2022
It is shown that the proposed novel robust training method, EWS, greatly improves the robustness against corrupted images as well as the accuracy on clean data, and is complementary to many state-of-the-art data augmentation approaches.
Defending Against Image Corruptions Through Adversarial Augmentations
- Computer ScienceArXiv
- 2021
This work proposes AdversarialAugment, a technique which optimizes the parameters of image-to-image models to generate adversarially corrupted augmented images and improves worst-case performance against `p-norm bounded perturbations on both CIFAR-10 and IMAGENET.
How Does Frequency Bias Affect the Robustness of Neural Image Classifiers against Common Corruption and Adversarial Perturbations?
- Computer ScienceArXiv
- 2022
This work proposes Jacobian frequency regularization for models’ Jacobians to have a larger ratio of low-frequency components and shows that biasing classifiers towards low (high)-frequency components can bring performance gain against high (low)-frequency corruption and adversarial perturbation, albeit with a tradeoff in performance for low ( high-frequency corruption.
NoisyMix: Boosting Model Robustness to Common Corruptions
- Computer Science
- 2022
NoisyMix is a novel training scheme that promotes stability as well as leverages noisy augmentations in input and feature space to improve both model robustness and in-domain accuracy and provides theory to understand implicit regularization and robustness of NoisyMix.
DEFENDING AGAINST IMAGE CORRUPTIONS THROUGH ADVERSARIAL AUGMENTATIONS
- Computer Science
- 2022
Classifiers trained using the proposed AdversarialAugment method in conjunction with prior methods improve upon the state-of-the-art on common image corruption benchmarks conducted in expectation on CIFAR-10-C and also improve worst-case performance against `p-norm bounded perturbations on both CIFar-10 and IMAGENET.
Adversarial amplitude swap towards robust image classifiers
- Computer Science, Environmental ScienceArXiv
- 2022
Results showed that adversarial amplitude images can serve as a better data augmentation method to achieve general robustness against both common corruptions and adversarial perturbations even in an adversarial training setup, and contributed to the understanding and the training of truly robust classifiers.
Improving robustness against common corruptions by covariate shift adaptation
- Computer ScienceNeurIPS
- 2020
It is argued that results with adapted statistics should be included whenever reporting scores in corruption benchmarks and other out-of-distribution generalization settings, and 32 samples are sufficient to improve the current state of the art for a ResNet-50 architecture.
ImageNet-Patch: A Dataset for Benchmarking Machine Learning Robustness against Adversarial Patches
- Computer ScienceArXiv
- 2022
This work proposes ImageNet-Patch, a dataset to benchmark machinelearning models against adversarial patches, a set of patches, optimized to generalize across different models, and readily applicable to ImageNet data after preprocessing them with affine transformations, enabling an approximate yet faster robustness evaluation.
References
SHOWING 1-10 OF 75 REFERENCES
Measuring Neural Net Robustness with Constraints
- Computer ScienceNIPS
- 2016
This work proposes metrics for measuring the robustness of a neural net and devise a novel algorithm for approximating these metrics based on an encoding of robustness as a linear program and generates more informative estimates of robusts metrics compared to estimates based on existing algorithms.
On Detecting Adversarial Perturbations
- Computer ScienceICLR
- 2017
It is shown empirically that adversarial perturbations can be detected surprisingly well even though they are quasi-imperceptible to humans.
Adversarially Robust Generalization Requires More Data
- Computer ScienceNeurIPS
- 2018
It is shown that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning.
Towards Evaluating the Robustness of Neural Networks
- Computer Science2017 IEEE Symposium on Security and Privacy (SP)
- 2017
It is demonstrated that defensive distillation does not significantly increase the robustness of neural networks, and three new attack algorithms are introduced that are successful on both distilled and undistilled neural networks with 100% probability are introduced.
Using Trusted Data to Train Deep Networks on Labels Corrupted by Severe Noise
- Computer ScienceNeurIPS
- 2018
It is demonstrated that robustness to label noise up to severe strengths can be achieved by using a set of trusted data with clean labels, and a loss correction that utilizes trusted examples in a data-efficient manner to mitigate the effects of label noise on deep neural network classifiers is proposed.
Improving the Robustness of Deep Neural Networks via Stability Training
- Computer Science2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)
- 2016
This paper presents a general stability training method to stabilize deep networks against small input distortions that result from various types of common image processing, such as compression, rescaling, and cropping.
Ground-Truth Adversarial Examples
- Computer ScienceArXiv
- 2017
Ground truths are constructed: adversarial examples with a provably-minimal distance from a given input point that can serve to assess the effectiveness of attack techniques and also of defense techniques, by computing the distance to the ground truths before and after the defense is applied, and measuring the improvement.
Robust Physical-World Attacks on Deep Learning Models
- Computer Science
- 2017
This work proposes a general attack algorithm,Robust Physical Perturbations (RP2), to generate robust visual adversarial perturbations under different physical conditions and shows that adversarial examples generated using RP2 achieve high targeted misclassification rates against standard-architecture road sign classifiers in the physical world under various environmental conditions, including viewpoints.
Adversarial Examples Are Not Easily Detected: Bypassing Ten Detection Methods
- Computer ScienceAISec@CCS
- 2017
It is concluded that adversarialExamples are significantly harder to detect than previously appreciated, and the properties believed to be intrinsic to adversarial examples are in fact not.
Distillation as a Defense to Adversarial Perturbations Against Deep Neural Networks
- Computer Science2016 IEEE Symposium on Security and Privacy (SP)
- 2016
The study shows that defensive distillation can reduce effectiveness of sample creation from 95% to less than 0.5% on a studied DNN, and analytically investigates the generalizability and robustness properties granted by the use of defensive Distillation when training DNNs.