• Corpus ID: 244708999

The Geometry of Adversarial Training in Binary Classification

@article{Bungert2021TheGO,
title={The Geometry of Adversarial Training in Binary Classification},
author={Leon Bungert and Nicol{\'a}s Garc{\'i}a Trillos and Ryan W. Murray},
journal={ArXiv},
year={2021},
volume={abs/2111.13613}
}
• Published 26 November 2021
• Mathematics, Computer Science
• ArXiv
We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type L+ (nonlocal) TV, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows…

Figures from this paper

Adversarial Classification: Necessary conditions and geometric flows
• Computer Science, Mathematics
ArXiv
• 2020
A version of adversarial classification where an adversary is empowered to corrupt data inputs up to some distance $\varepsilon$ is studied, using tools from variational analysis to derive a geometric evolution equation which can be used to track the change in classification boundaries as $\vARPSilon$ varies.
• Computer Science
• 2022
This paper exposes some pathological behaviors to the adversarial problem, and shows that no convex surrogate loss can be consistent or calibrated in this context, and identifies suﬁcient and necessary conditions for a surrogate loss to be calibrated in both the adversary and standard settings.
The Consistency of Adversarial Training for Binary Classification
• Computer Science
ArXiv
• 2022
Which supremum-based surrogates are consistent for distributions absolutely continuous with respect to Lebesgue measure in binary classification is characterized and quantitative bounds relating adversarial surrogate risks to the adversarial classification risk are obtained.
Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification
This work proves and existence, regularity, and minimax theorems for adversarial surrogate risks for adversarian surrogate risks and extends previously known existence and minimx theorem for the adversarial classification risk to surrogate risks.
Probabilistically Robust Learning: Balancing Average- and Worst-case Performance
• Computer Science
ArXiv
• 2022
A framework called probabilistic robustness is proposed that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case by enforcing robustness to most rather than to all perturbations.
Eikonal depth: an optimal control approach to statistical depths
• Mathematics
ArXiv
• 2022
A new type of globally defined statistical depth is proposed, based upon control theory and eikonal equations, which measures the smallest amount of probability density that has to be passed through in a path to points outside the support of the distribution: for example spatial infinity.

References

SHOWING 1-10 OF 65 REFERENCES
Robustness via Curvature Regularization, and Vice Versa
• Computer Science
2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
• 2019
It is shown in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network.
Adversarial Classification: Necessary conditions and geometric flows
• Computer Science, Mathematics
ArXiv
• 2020
A version of adversarial classification where an adversary is empowered to corrupt data inputs up to some distance $\varepsilon$ is studied, using tools from variational analysis to derive a geometric evolution equation which can be used to track the change in classification boundaries as $\vARPSilon$ varies.
Lower Bounds on Adversarial Robustness from Optimal Transport
• Computer Science
NeurIPS
• 2019
While progress has been made in understanding the robustness of machine learning classifiers to test-time adversaries (evasion attacks), fundamental questions remain unresolved. In this paper, we use
Learned convex regularizers for inverse problems.
• Mathematics
• 2020
This work shows that the optimal solution to the variational problem converges to the ground-truth if the penalty parameter decays sub-linearly with respect to the norm of the noise and proves the existence of a subgradient-based algorithm that leads to monotonically decreasing error in the parameter space with iterations.
The Many Faces of Adversarial Risk
The results generalize and deepen recently discovered connections between optimal transport and adversarial robustness and reveal new connections to Choquet capacities and game theory.
Improved robustness to adversarial examples using Lipschitz regularization of the loss
• Computer Science, Mathematics
ArXiv
• 2018
This work augments AT with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10, and obtains verifiable average case and worst case robustness guarantees.
• Computer Science
2015 IEEE International Conference on Data Mining
• 2015
A family of gradient regularization methods that effectively penalize the gradient of loss function w.r.t. inputs are developed and achieved the best accuracy on MNIST data (without data augmentation) and competitive performance on CIFAR-10 data.
Adversarial Risk via Optimal Transport and Optimal Couplings
• Computer Science
IEEE Transactions on Information Theory
• 2021
This paper presents a new and simple approach to show that the optimal adversarial risk for binary classification with \$0-1 loss function is completely characterized by an optimal transport cost between the probability distributions of the two classes, for a suitably defined cost function.
Towards Deep Learning Models Resistant to Adversarial Attacks
• Computer Science
ICLR
• 2018
This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee.
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
• Computer Science
AAAI
• 2018
It is demonstrated that regularizing input gradients makes them more naturally interpretable as rationales for model predictions, and also exhibits robustness to transferred adversarial examples generated to fool all of the other models.