Corpus ID: 159041190

Adversarially robust transfer learning

@article{Shafahi2020AdversariallyRT,
  title={Adversarially robust transfer learning},
  author={Ali Shafahi and Parsa Saadatpanah and Chen Zhu and Amin Ghiasi and Christoph Studer and David W. Jacobs and Tom Goldstein},
  journal={ArXiv},
  year={2020},
  volume={abs/1905.08232}
}
Transfer learning, in which a network is trained on one task and re-purposed on another, is often used to produce neural network classifiers when data is scarce or full-scale training is too costly. When the goal is to produce a model that is not only accurate but also adversarially robust, data scarcity and computational limitations become even more cumbersome. We consider robust transfer learning, in which we transfer not only performance but also robustness from a source model to a target… Expand
CARTL: Cooperative Adversarially-Robust Transfer Learning
TLDR
This work proposes a novel cooperative adversarially-robust transfer learning (CARTL), which improves the accuracy on the target domain but degrades the inherited robustness of the target model by pre-training the model via feature distance minimization and fine- Tuning the pre-trained model with non-expansive fine-tuning for target domain tasks. Expand
Improving the Adversarial Robustness of Transfer Learning via Noisy Feature Distillation
TLDR
This work proposes noisy feature distillation, a new transfer learning method that trains a network from random initialization while achieving clean-data performance competitive with fine-tuning, and is shown empirically to significantly improve the robustness compared to fine- Tuning. Expand
Adversarially-Trained Deep Nets Transfer Better
TLDR
It is demonstrated that adversarially- trained models transfer better across new domains than naturally-trained models, even though it's known that these models do not generalize as well as naturally-training models on the source domain. Expand
Adversarially-Trained Deep Nets Transfer Better: Illustration on Image Classification
TLDR
It is demonstrated that adversarially-trained models transfer better than non-adversarially -trained models, especially if only limited data are available for the new domain task. Expand
Renofeation: A Simple Transfer Learning Method for Improved Adversarial Robustness
TLDR
This work proposes noisy feature distillation, a new transfer learning method that trains a network from random initialization while achieving clean-data performance competitive with fine-tuning. Expand
Robust Few-Shot Learning with Adversarially Queried Meta-Learners
TLDR
This work adapts adversarial training for meta-learning, it adapt robust architectural features to small networks for metalearning, it test pre-processing defenses as an alternative to adversarial Training for Meta- learning, and it investigates the advantages of robust meta- learning over robust transfer-learning for few-shot tasks. Expand
On Fast Adversarial Robustness Adaptation in Model-Agnostic Meta-Learning
TLDR
A general but easily-optimized robustness-regularized meta-learning framework, which allows the use of unlabeled data augmentation, fast adversarial attack generation, and computationally-light fine-tuning, and for the first time shows that the auxiliary contrastive learning task can enhance the adversarial robustness of MAML. Expand
Robust Learning Meets Generative Models: Can Proxy Distributions Improve Adversarial Robustness?
TLDR
A robust discrimination approach is proposed, which measures the distinguishability of synthetic and real samples under adversarial perturbations and accurately predicts the robustness transfer from different proxy distributions. Expand
Adversarial Robustness for Unsupervised Domain Adaptation
  • Muhammad Awais, Fengwei Zhou, +4 authors Zhenguo Li
  • Computer Science
  • ArXiv
  • 2021
TLDR
This work leverages intermediate representations learned by multiple robust ImageNet models to improve the robustness of UDA models and works by aligning the features of the UDA model with the robust features learned by ImageNet pre-trained models along with domain adaptation training. Expand
How and When Adversarial Robustness Transfers in Knowledge Distillation?
  • Rulin Shao, Jinfeng Yi, Pin-Yu Chen, Cho-Jui Hsieh
  • Computer Science
  • ArXiv
  • 2021
TLDR
This paper shows that standard KD training fails to preserve adversarial robustness, and proposes KD with input gradient alignment (KDIGA) for remedy, and proves that the student model using the proposed KDIGA can achieve at least the same certified robustness as the teacher model. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 27 REFERENCES
Adversarially Robust Generalization Requires More Data
TLDR
It is shown that already in a simple natural data model, the sample complexity of robust learning can be significantly larger than that of "standard" learning. Expand
Using Pre-Training Can Improve Model Robustness and Uncertainty
TLDR
It is shown that although pre-training may not improve performance on traditional classification metrics, it improves model robustness and uncertainty estimates and surpasses the state-of-the-art in adversarial robustness. Expand
Towards Evaluating the Robustness of Neural Networks
TLDR
It is demonstrated that defensive distillation does not significantly increase the robustness of neural networks, and three new attack algorithms are introduced that are successful on both distilled and undistilled neural networks with 100% probability are introduced. Expand
Adversarial Machine Learning at Scale
TLDR
This research applies adversarial training to ImageNet and finds that single-step attacks are the best for mounting black-box attacks, and resolution of a "label leaking" effect that causes adversarially trained models to perform better on adversarial examples than on clean examples. Expand
Adversarial Training for Free!
TLDR
This work presents an algorithm that eliminates the overhead cost of generating adversarial examples by recycling the gradient information computed when updating model parameters, and achieves comparable robustness to PGD adversarial training on the CIFAR-10 and CIFar-100 datasets at negligible additional cost compared to natural training. Expand
Ensemble Adversarial Training: Attacks and Defenses
TLDR
This work finds that adversarial training remains vulnerable to black-box attacks, where perturbations computed on undefended models are transferred to a powerful novel single-step attack that escapes the non-smooth vicinity of the input data via a small random step. Expand
Explaining and Harnessing Adversarial Examples
TLDR
It is argued that the primary cause of neural networks' vulnerability to adversarial perturbation is their linear nature, supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Expand
Towards Deep Learning Models Resistant to Adversarial Attacks
TLDR
This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee. Expand
How transferable are features in deep neural networks?
TLDR
This paper quantifies the generality versus specificity of neurons in each layer of a deep convolutional neural network and reports a few surprising results, including that initializing a network with transferred features from almost any number of layers can produce a boost to generalization that lingers even after fine-tuning to the target dataset. Expand
DeepFool: A Simple and Accurate Method to Fool Deep Neural Networks
TLDR
The DeepFool algorithm is proposed to efficiently compute perturbations that fool deep networks, and thus reliably quantify the robustness of these classifiers, and outperforms recent methods in the task of computing adversarial perturbation and making classifiers more robust. Expand
...
1
2
3
...