• Corpus ID: 244708999

The Geometry of Adversarial Training in Binary Classification

@article{Bungert2021TheGO,
  title={The Geometry of Adversarial Training in Binary Classification},
  author={Leon Bungert and Nicol{\'a}s Garc{\'i}a Trillos and Ryan W. Murray},
  journal={ArXiv},
  year={2021},
  volume={abs/2111.13613}
}
We establish an equivalence between a family of adversarial training problems for non-parametric binary classification and a family of regularized risk minimization problems where the regularizer is a nonlocal perimeter functional. The resulting regularized risk minimization problems admit exact convex relaxations of the type L+ (nonlocal) TV, a form frequently studied in image analysis and graph-based learning. A rich geometric structure is revealed by this reformulation which in turn allows… 

Figures from this paper

Adversarial Classification: Necessary conditions and geometric flows
TLDR
A version of adversarial classification where an adversary is empowered to corrupt data inputs up to some distance $\varepsilon$ is studied, using tools from variational analysis to derive a geometric evolution equation which can be used to track the change in classification boundaries as $\vARPSilon$ varies.
Towards Consistency in Adversarial Classification
TLDR
This paper exposes some pathological behaviors to the adversarial problem, and shows that no convex surrogate loss can be consistent or calibrated in this context, and identifies suficient and necessary conditions for a surrogate loss to be calibrated in both the adversary and standard settings.
The Consistency of Adversarial Training for Binary Classification
TLDR
Which supremum-based surrogates are consistent for distributions absolutely continuous with respect to Lebesgue measure in binary classification is characterized and quantitative bounds relating adversarial surrogate risks to the adversarial classification risk are obtained.
Existence and Minimax Theorems for Adversarial Surrogate Risks in Binary Classification
TLDR
This work proves and existence, regularity, and minimax theorems for adversarial surrogate risks for adversarian surrogate risks and extends previously known existence and minimx theorem for the adversarial classification risk to surrogate risks.
Probabilistically Robust Learning: Balancing Average- and Worst-case Performance
TLDR
A framework called probabilistic robustness is proposed that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case by enforcing robustness to most rather than to all perturbations.
Eikonal depth: an optimal control approach to statistical depths
TLDR
A new type of globally defined statistical depth is proposed, based upon control theory and eikonal equations, which measures the smallest amount of probability density that has to be passed through in a path to points outside the support of the distribution: for example spatial infinity.

References

SHOWING 1-10 OF 65 REFERENCES
Robustness via Curvature Regularization, and Vice Versa
TLDR
It is shown in particular that adversarial training leads to a significant decrease in the curvature of the loss surface with respect to inputs, leading to a drastically more "linear" behaviour of the network.
Adversarial Classification: Necessary conditions and geometric flows
TLDR
A version of adversarial classification where an adversary is empowered to corrupt data inputs up to some distance $\varepsilon$ is studied, using tools from variational analysis to derive a geometric evolution equation which can be used to track the change in classification boundaries as $\vARPSilon$ varies.
Lower Bounds on Adversarial Robustness from Optimal Transport
While progress has been made in understanding the robustness of machine learning classifiers to test-time adversaries (evasion attacks), fundamental questions remain unresolved. In this paper, we use
Learned convex regularizers for inverse problems.
TLDR
This work shows that the optimal solution to the variational problem converges to the ground-truth if the penalty parameter decays sub-linearly with respect to the norm of the noise and proves the existence of a subgradient-based algorithm that leads to monotonically decreasing error in the parameter space with iterations.
The Many Faces of Adversarial Risk
TLDR
The results generalize and deepen recently discovered connections between optimal transport and adversarial robustness and reveal new connections to Choquet capacities and game theory.
Improved robustness to adversarial examples using Lipschitz regularization of the loss
TLDR
This work augments AT with worst case adversarial training (WCAT) which improves adversarial robustness by 11% over the current state-of-the-art result in the $\ell_2$ norm on CIFAR-10, and obtains verifiable average case and worst case robustness guarantees.
A Unified Gradient Regularization Family for Adversarial Examples
TLDR
A family of gradient regularization methods that effectively penalize the gradient of loss function w.r.t. inputs are developed and achieved the best accuracy on MNIST data (without data augmentation) and competitive performance on CIFAR-10 data.
Adversarial Risk via Optimal Transport and Optimal Couplings
TLDR
This paper presents a new and simple approach to show that the optimal adversarial risk for binary classification with $0-1 loss function is completely characterized by an optimal transport cost between the probability distributions of the two classes, for a suitably defined cost function.
Towards Deep Learning Models Resistant to Adversarial Attacks
TLDR
This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee.
Improving the Adversarial Robustness and Interpretability of Deep Neural Networks by Regularizing their Input Gradients
TLDR
It is demonstrated that regularizing input gradients makes them more naturally interpretable as rationales for model predictions, and also exhibits robustness to transferred adversarial examples generated to fool all of the other models.
...
...