Scaleable input gradient regularization for adversarial robustness

@article{Finlay2019ScaleableIG,
  title={Scaleable input gradient regularization for adversarial robustness},
  author={Chris Finlay and Adam M. Oberman},
  journal={ArXiv},
  year={2019},
  volume={abs/1905.11468}
}
In this work we revisit gradient regularization for adversarial robustness with some new ingredients. First, we derive new per-image theoretical robustness bounds based on local gradient information. These bounds strongly motivate input gradient regularization. Second, we implement a scaleable version of input gradient regularization which avoids double backpropagation: adversarially robust ImageNet models are trained in 33 hours on four consumer grade GPUs. Finally, we show experimentally and… Expand
Input Hessian Regularization of Neural Networks
TLDR
It is proved that the Hessian operator norm relates to the ability of a neural network to withstand an adversarial attack and, furthermore, that it increases the robustness of neural networks over input gradient regularization. Expand
Improving Gradient Regularization using Complex-Valued Neural Networks
TLDR
Experimental results show that the performance of gradient regularized CVNN surpasses that of real-valued neural networks with comparable storage and computational complexity and that the properties of the CVNN parameter derivatives resist decrease of performance on the standard objective that is caused by competition with the gradient regularization objective. Expand
Adversarial Robustness Through Local Lipschitzness
TLDR
The results show that having a small Lipschitz constant correlates with achieving high clean and robust accuracy, and therefore, the smoothness of the classifier is an important property to consider in the context of adversarial examples. Expand
Adversarial Boot Camp: label free certified robustness in one epoch
TLDR
This work presents a deterministic certification approach which results in a certifiably robust model based on an equivalence between training with a particular regularized loss, and the expected values of Gaussian averages. Expand
On the human-recognizability phenomenon of adversarially trained deep image classifiers
TLDR
This work demonstrates that state-of-theart methods for adversarial training incorporate two terms – one that orients the decision boundary via minimizing the expected loss, and another that induces smoothness of the classifier’s decision surface by penalizing the local Lipschitz constant. Expand
Deterministic Gaussian Averaged Neural Networks
TLDR
A deterministic method to compute the Gaussian average of neural networks used in regression and classification is presented, comparable to known stochastic methods such as randomized smoothing, but requires only a single model evaluation during inference. Expand
A principled approach for generating adversarial images under non-smooth dissimilarity metrics
TLDR
This work proposes an attack methodology not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarity metric with a closed proximal form, and eliminates the differentiability requirement of the metric. Expand
Are Adversarial Examples Created Equal? A Learnable Weighted Minimax Risk for Robustness under Non-uniform Attacks
TLDR
A weighted minimax risk optimization that defends against non-uniform attacks, achieving robustness against adversarial examples under perturbed test data distributions and significantly improves state-of-the-art adversarial accuracy under non- uniform attacks. Expand
Min-Max Optimization without Gradients: Convergence and Applications to Black-Box Evasion and Poisoning Attacks
TLDR
A principled optimization framework, integrating a zeroth-order (ZO) gradient estimator with an alternating projected stochastic gradient descent-ascent method, where the former only requires a small number of function queries and the later needs just one-step descent/ascent update. Expand
Defending Adversarial Attacks without Adversarial Attacks in Deep Reinforcement Learning
TLDR
A new policy distillation loss that consists of a prescription gap maximization loss aiming at simultaneously maximizing the likelihood of the action selected by the teacher policy and the entropy over the remaining actions and a Jacobian regularization loss that minimizes the magnitude of Jacobian with respect to the input state is proposed. Expand
...
1
2
3
...

References

SHOWING 1-10 OF 71 REFERENCES
Adversarially Robust Training through Structured Gradient Regularization
We propose a novel data-dependent structured gradient regularizer to increase the robustness of neural networks vis-a-vis adversarial perturbations. Our regularizer can be derived as a controlledExpand
Unifying Adversarial Training Algorithms with Data Gradient Regularization
TLDR
The proposed DataGrad framework, which can be viewed as a deep extension of the layerwise contractive autoencoder penalty, cleanly simplifies prior work and easily allows extensions such as adversarial training with multitask cues. Expand
Adversarial Vulnerability of Neural Networks Increases With Input Dimension
TLDR
This work shows that adversarial vulnerability increases with the gradients of the training objective when seen as a function of the inputs, and rediscover and generalize double-backpropagation, a technique that penalizes large gradients in the loss surface to reduce adversarialulnerability and increase generalization performance. Expand
Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers
TLDR
It is demonstrated through extensive experimentation that this method consistently outperforms all existing provably $\ell-2$-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable $\ell_ 2$-defenses. Expand
Scaling provable adversarial defenses
TLDR
This paper presents a technique for extending these training procedures to much more general networks, with skip connections and general nonlinearities, and shows how to further improve robust error through cascade models. Expand
Lipschitz-Margin Training: Scalable Certification of Perturbation Invariance for Deep Neural Networks
TLDR
From the relationship between the Lipschitz constants and prediction margins, a computationally efficient calculation technique is presented to lower-bound the size of adversarial perturbations that can deceive networks, and that is widely applicable to various complicated networks. Expand
Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning
TLDR
A new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input that achieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10. Expand
Improving DNN Robustness to Adversarial Attacks using Jacobian Regularization
TLDR
This work suggests a theoretically inspired novel approach to improve the networks' robustness using the Frobenius norm of the Jacobian of the network, which is applied as post-processing, after regular training has finished and demonstrates empirically that it leads to enhanced robustness results with a minimal change in the original network's accuracy. Expand
Stabilizing Training of Generative Adversarial Networks through Regularization
TLDR
This work proposes a new regularization approach with low computational cost that yields a stable GAN training procedure and demonstrates the effectiveness of this regularizer accross several architectures trained on common benchmark image generation tasks. Expand
Robustness via Curvature Regularization, and Vice Versa
TLDR
It is shown in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network. Expand
...
1
2
3
4
5
...