@article{Finlay2019ScaleableIG,
author={Chris Finlay and Adam M. Oberman},
journal={ArXiv},
year={2019},
volume={abs/1905.11468}
}
• Published 2019
• Computer Science, Mathematics
• ArXiv
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs. Finally, we show experimentally and… Expand
Input Hessian Regularization of Neural Networks
• Computer Science, Mathematics
• ArXiv
• 2020
It is proved that the Hessian operator norm relates to the ability of a neural network to withstand an adversarial attack and, furthermore, that it increases the robustness of neural networks over input gradient regularization. Expand
Improving Gradient Regularization using Complex-Valued Neural Networks
• Computer Science
• ICML
• 2021
Experimental results show that the performance of gradient regularized CVNN surpasses that of real-valued neural networks with comparable storage and computational complexity and that the properties of the CVNN parameter derivatives resist decrease of performance on the standard objective that is caused by competition with the gradient regularization objective. Expand
• Computer Science, Mathematics
• ArXiv
• 2020
The results show that having a small Lipschitz constant correlates with achieving high clean and robust accuracy, and therefore, the smoothness of the classifier is an important property to consider in the context of adversarial examples. Expand
Adversarial Boot Camp: label free certified robustness in one epoch
• Computer Science, Mathematics
• ArXiv
• 2020
This work presents a deterministic certification approach which results in a certifiably robust model based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. Expand
On the human-recognizability phenomenon of adversarially trained deep image classifiers
• Computer Science
• ArXiv
• 2021
This work demonstrates that state-of-theart methods for adversarial training incorporate two terms – one that orients the decision boundary via minimizing the expected loss, and another that induces smoothness of the classifier’s decision surface by penalizing the local Lipschitz constant. Expand
Deterministic Gaussian Averaged Neural Networks
• Computer Science, Mathematics
• ArXiv
• 2020
A deterministic method to compute the Gaussian average of neural networks used in regression and classification is presented, comparable to known stochastic methods such as randomized smoothing, but requires only a single model evaluation during inference. Expand
A principled approach for generating adversarial images under non-smooth dissimilarity metrics
• Computer Science, Mathematics
• AISTATS
• 2020
This work proposes an attack methodology not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarity metric with a closed proximal form, and eliminates the differentiability requirement of the metric. Expand
Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
• Computer Science
• AAAI
• 2021
A weighted minimax risk optimization that defends against non-uniform attacks, achieving robustness against adversarial examples under perturbed test data distributions and significantly improves state-of-the-art adversarial accuracy under non- uniform attacks. Expand
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
A principled optimization framework, integrating a zeroth-order (ZO) gradient estimator with an alternating projected stochastic gradient descent-ascent method, where the former only requires a small number of function queries and the later needs just one-step descent/ascent update. Expand
• Computer Science, Mathematics
• ArXiv
• 2020
A new policy distillation loss that consists of a prescription gap maximization loss aiming at simultaneously maximizing the likelihood of the action selected by the teacher policy and the entropy over the remaining actions and a Jacobian regularization loss that minimizes the magnitude of Jacobian with respect to the input state is proposed. Expand

#### References

SHOWING 1-10 OF 71 REFERENCES
• Computer Science, Mathematics
• ArXiv
• 2018
We propose a novel data-dependent structured gradient regularizer to increase the robustness of neural networks vis-a-vis adversarial perturbations. Our regularizer can be derived as a controlledExpand
• Mathematics, Computer Science
• Neural Computation
• 2017
The proposed DataGrad framework, which can be viewed as a deep extension of the layerwise contractive autoencoder penalty, cleanly simplifies prior work and easily allows extensions such as adversarial training with multitask cues. Expand
Adversarial Vulnerability of Neural Networks Increases With Input Dimension
• Mathematics, Computer Science
• ArXiv
• 2018
This work shows that adversarial vulnerability increases with the gradients of the training objective when seen as a function of the inputs, and rediscover and generalize double-backpropagation, a technique that penalizes large gradients in the loss surface to reduce adversarialulnerability and increase generalization performance. Expand
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
It is demonstrated through extensive experimentation that this method consistently outperforms all existing provably $\ell-2$-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable $\ell_ 2$-defenses. Expand
• Computer Science, Mathematics
• NeurIPS
• 2018
This paper presents a technique for extending these training procedures to much more general networks, with skip connections and general nonlinearities, and shows how to further improve robust error through cascade models. Expand
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
• Computer Science, Mathematics
• NeurIPS
• 2018
From the relationship between the Lipschitz constants and prediction margins, a computationally efficient calculation technique is presented to lower-bound the size of adversarial perturbations that can deceive networks, and that is widely applicable to various complicated networks. Expand
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
• Computer Science, Mathematics
• IEEE Transactions on Pattern Analysis and Machine Intelligence
• 2019
A new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input that achieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10. Expand
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization
• Computer Science, Mathematics
• ECCV
• 2018
This work suggests a theoretically inspired novel approach to improve the networks' robustness using the Frobenius norm of the Jacobian of the network, which is applied as post-processing, after regular training has finished and demonstrates empirically that it leads to enhanced robustness results with a minimal change in the original network's accuracy. Expand
Stabilizing Training of Generative Adversarial Networks through Regularization
• Computer Science, Mathematics
• NIPS
• 2017
This work proposes a new regularization approach with low computational cost that yields a stable GAN training procedure and demonstrates the effectiveness of this regularizer accross several architectures trained on common benchmark image generation tasks. Expand
Robustness via Curvature Regularization, and Vice Versa
• Computer Science, Mathematics
• 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 2019
It is shown in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network. Expand