• Corpus ID: 240288565

Adversarial Robustness with Semi-Infinite Constrained Learning

@inproceedings{Robey2021AdversarialRW,
  title={Adversarial Robustness with Semi-Infinite Constrained Learning},
  author={Alexander Robey and Luiz F. O. Chamon and George J. Pappas and Hamed Hassani and Alejandro Ribeiro},
  booktitle={Neural Information Processing Systems},
  year={2021}
}
Despite strong performance in numerous applications, the fragility of deep learning to input perturbations has raised serious questions about its use in safety-critical domains. While adversarial training can mitigate this issue in practice, state-of-the-art methods are increasingly application-dependent, heuristic in nature, and suffer from fundamental trade-offs between nominal performance and robustness. Moreover, the problem of finding worst-case perturbations is non-convex and… 
[PDF] Semantic Reader
21 Citations
Highly Influential Citations
1
Background Citations
14
Methods Citations
6

Figures and Tables from this paper

21 Citations

Probabilistically Robust Learning: Balancing Average- and Worst-case Performance

A framework called probabilistic robustness is proposed that bridges the gap between the accurate, yet brittle average case and the robust, yet conservative worst case by enforcing robustness to most rather than to all perturbations.

Towards Adversarial Robustness with Multidimensional Perturbations via Contrastive Learning

This work proposes a novel approach to adversarial robustness, which establishes on the insights from min-max optimization that more powerful adversarial perturbations lead to more robust defense.

A Minimax Approach Against Multi-Armed Adversarial Attacks Detection

This work proposes a solution that aggregates the soft-probability outputs of multiple pre-trained detectors according to a minimax approach and shows that this aggregation consistently outperforms individual state-of-the-art detectors against multi-armed adversarial attacks, making it an effective solution to improve the resilience of available methods.

Robustness Against Adversarial Attacks in Neural Networks Using Incremental Dissipativity

This letter proposes an incremental dissipativity-based robustness certificate for neural networks in the form of a linear matrix inequality for each layer, and proposes a sufficient spectral norm bound for this certificate which is scalable to neural networks with multiple layers.

Internal Wasserstein Distance for Adversarial Attack and Defense

A new internal Wasserstein distance (IWD) is proposed to capture the semantic similarity of two samples, and thus it helps to obtain larger perturbations than currently used metrics such as the $\ell_p$ distance.

Adversarial Attacks and Defense for Non-Parametric Two-Sample Tests

This paper systematically uncovers the failure mode of non-parametric TSTs through adversarial attacks and then pro-poses corresponding defense strategies to robustify and enable TST-agnostic attacks.

Constrained Learning With Non-Convex Losses

This paper establishes a constrained counterpart to classical learning theory, enabling the explicit use of constraints in learning by learning in the empirical dual domain, where constrained statistical learning problems become unconstrained and deterministic.

Towards Principled Disentanglement for Domain Generalization

This work formalizes the OOD generalization problem as constrained optimization as Disentanglement-constrained Domain Generalization (DDG), and proposes a primal-dual algorithm for joint representation disentanglements and domain generalization.

Toward Certified Robustness Against Real-World Distribution Shifts

This work considers the problem of certifying the robustness of deep neural networks against real-world distribution shifts by considering a neural-symbolic verification framework in which generative models are trained to learn perturbations from data and specifications are defined with respect to the output of these learned models.

Learning Robust Kernel Ensembles with Kernel Average Pooling

KAP, a neural network building block that applies the mean filter along the kernel dimension of the layer activation tensor, is introduced, showing that ensembles of kernels with similar functionality naturally emerge in convolutional neural networks equipped with KAP and trained with backpropagation.

123 References

Precise Tradeoffs in Adversarial Training for Linear Regression

A precise and comprehensive understanding of the role of adversarial training in the context of linear regression with Gaussian features is provided and the fundamental tradeoff between the accuracies achievable by any algorithm regardless of computational power or size of the training data is characterized.

Provable defenses against adversarial examples via the convex outer adversarial polytope

A method to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations, and it is shown that the dual problem to this linear program can be represented itself as a deep network similar to the backpropagation network, leading to very efficient optimization approaches that produce guaranteed bounds on the robust loss.

Adversarial Learning Guarantees for Linear Hypotheses and Neural Networks

This work focuses on the problem of understanding generalization in adversarial settings, via the lens of Rademacher complexity, and gives upper and lower bounds for the adversarial empirical RadEmacher complexity of linear hypotheses with adversarial perturbations measured in $l_r$-norm for an arbitrary $r \geq 1$.

Theoretical Analysis of Adversarial Learning: A Minimax Approach

A general theoretical method for analyzing the risk bound in the presence of adversaries and derives the adversarial risk bounds for SVMs, deep neural networks, and PCA, and these bounds have two data-dependent terms, which can be optimized for achieving adversarial robustness.

Towards Deep Learning Models Resistant to Adversarial Attacks

This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee.

Achieving Robustness in the Wild via Adversarial Mixing With Disentangled Representations

This paper uses the disentangled latent representations computed by a StyleGAN model to generate perturbations of an image that are similar to real-world variations and trains models to be invariant to these perturbation.

Stochastic Activation Pruning for Robust Adversarial Defense

Stochastic Activation Pruning (SAP) is proposed, a mixed strategy for adversarial defense that prunes a random subset of activations (preferentially pruning those with smaller magnitude) and scales up the survivors to compensate.

Rademacher Complexity for Adversarially Robust Generalization

For binary linear classifiers, it is shown that the adversarial Rademacher complexity is never smaller than its natural counterpart, and it has an unavoidable dimension dependence, unless the weight vector has bounded $\ell_1$ norm.

Understanding adversarial training: Increasing local stability of supervised models through robust optimization

Certifying Some Distributional Robustness with Principled Adversarial Training

This work provides a training procedure that augments model parameter updates with worst-case perturbations of training data and efficiently certify robustness for the population loss by considering a Lagrangian penalty formulation of perturbing the underlying data distribution in a Wasserstein ball.
...