Corpus ID: 235421591

Expected Tight Bounds for Robust Training

  title={Expected Tight Bounds for Robust Training},
  author={Salman Alsubaihi and Adel Bibi and Modar Alfadly and Abdullah Hamdi and Bernard Ghanem},
Training deep neural networks that are robust to norm-bounded adversarial attacks remains an elusive problem. While exact and inexact verification-based methods are generally too expensive to train large networks, it was demonstrated that bounded input intervals can be inexpensively propagated from a layer to another through deep networks. This interval bound propagation approach (IBP) not only has improved both robustness and certified accuracy but was the first to be employed on large/deep… Expand

Figures and Tables from this paper


On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
This work shows how a simple bounding technique, interval bound propagation (IBP), can be exploited to train large provably robust neural networks that beat the state-of-the-art in verified accuracy and allows the largest model to be verified beyond vacuous bounds on a downscaled version of ImageNet. Expand
Towards Deep Learning Models Resistant to Adversarial Attacks
This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee. Expand
The mnist database of handwritten digits
Disclosed is an improved articulated bar flail having shearing edges for efficiently shredding materials. An improved shredder cylinder is disclosed with a plurality of these flails circumferentiallyExpand
Learning Multiple Layers of Features from Tiny Images
It is shown how to train a multi-layer generative model that learns to extract meaningful features which resemble those found in the human visual cortex, using a novel parallelization algorithm to distribute the work among multiple machines connected on a network. Expand
Evaluating the Robustness of Neural Networks: An Extreme Value Theory Approach
This paper provides a theoretical justification for converting robustness analysis into a local Lipschitz constant estimation problem, and proposes to use the Extreme Value Theory for efficient evaluation, which yields a novel robustness metric called CLEVER, which is short for Cross LPschitz Extreme Value for nEtwork Robustness. Expand
Distilling the Knowledge in a Neural Network
This work shows that it can significantly improve the acoustic model of a heavily used commercial system by distilling the knowledge in an ensemble of models into a single model and introduces a new type of ensemble composed of one or more full models and many specialist models which learn to distinguish fine-grained classes that the full models confuse. Expand
Evaluating Robustness of Neural Networks with Mixed Integer Programming
Verification of piecewise-linear neural networks as a mixed integer program that is able to certify more samples than the state-of-the-art and find more adversarial examples than a strong first-order attack for every network. Expand
Robustness May Be at Odds with Accuracy
It is shown that there may exist an inherent tension between the goal of adversarial robustness and that of standard generalization, and it is argued that this phenomenon is a consequence of robust classifiers learning fundamentally different feature representations than standard classifiers. Expand
Analytic Expressions for Probabilistic Moments of PL-DNN with Gaussian Input
This paper derives exact analytic expressions for the first and second moments of a small piecewise linear (PL) network (Affine, ReLU, Affine) subject to general Gaussian input and shows how these expressions can be used to systematically construct targeted and non-targeted adversarial attacks. Expand
Differentiable Abstract Interpretation for Provably Robust Neural Networks
Several abstract transformers which balance efficiency with precision are presented and it is shown these can be used to train large neural networks that are certifiably robust to adversarial perturbations. Expand