Adversarial Training is Not Ready for Robot Learning

  title={Adversarial Training is Not Ready for Robot Learning},
  author={Mathias Lechner and Ramin M. Hasani and Radu Grosu and Daniela Rus and Thomas A. Henzinger},
  journal={2021 IEEE International Conference on Robotics and Automation (ICRA)},
Adversarial training is an effective method to train deep learning models that are resilient to norm-bounded perturbations, with the cost of nominal performance drop. While adversarial training appears to enhance the robustness and safety of a deep model deployed in open-world decision-critical applications, counterintuitively, it induces undesired behaviors in robot learning settings. In this paper, we show theoretically and experimentally that neural controllers obtained via adversarial… 

Figures and Tables from this paper

Revisiting the Adversarial Robustness-Accuracy Tradeoff in Robot Learning

This work revisits the robustness-accuracy trade-off in robot learning through systematically analyzing if recent advances in robust training methods and theory in conjunction with adversarial robot learning, are capable of making adversarial training suitable for real-world robot applications.

Adversarially Regularized Policy Learning Guided by Trajectory Optimization

The proposed approach controls the smoothness (local Lipschitz continuity) of the neural control policies by stabilizing the output control with respect to the worst-case perturbation to the input state.

Better Safe Than Sorry: Preventing Delusive Adversaries with Adversarial Training

It is shown that minimizing adversarial risk on the perturbed data is equivalent to optimizing an upper bound of natural risk onThe original data, which implies that adversarial training can serve as a principled defense against delusive attacks.

Latent Imagination Facilitates Zero-Shot Transfer in Autonomous Racing

This paper investigates how model-based agents capable of learning in imagination substantially outperform model-free agents with respect to performance, sample efficiency, successful task completion, and generalization in real-world autonomous vehicle control tasks, where advanced model- free deep RL algorithms fail.

Rethink the Adversarial Scenario-based Safety Testing of Robots: the Comparability and Optimal Aggressiveness

This paper disputes the above intuition by introducing an impossibility theorem that provably shows all safety testing algorithms of the aforementioned difference perform equally well with the same expected sampling efficiency.

Interactive Analysis of CNN Robustness

Perturber is a web‐based application that allows users to instantaneously explore how CNN activations and predictions evolve when a 3D input scene is interactively perturbed, and replicate users’ insights with other CNN architectures and input images, yielding new insights about the vulnerability of adversarially trained models.

BarrierNet: A Safety-Guaranteed Layer for Neural Networks

These novel safety layers, termed a BarrierNet, can be used in conjunction with any neural network-based controller, and can be trained by gradient descent, which allows the safety constraints of a neural controller be adaptable to changing environments.

Causal Navigation by Continuous-time Neural Networks

The results demonstrate that causal continuous-time deep models can perform robust navigation tasks, where advanced recurrent models fail, and learn complex causal control representations directly from raw visual inputs and scale to solve a variety of tasks using imitation learning.

Beyond Robustness: A Taxonomy of Approaches towards Resilient Multi-Robot Systems

This survey article analyzed how resilience is achieved in networks of agents and multirobot systems that are able to overcome adversity by leveraging system-wide complementarity, diversity, and redundancy—often involving a reconfiguration of robotic capabilities to provide some key ability that was not present in the system a priori.

Sparse Flows: Pruning Continuous-depth Models

This work designs a framework to decipher the internal dynamics of these continuous depth models by pruning their network architectures, and empirical results suggest that pruning improves generalization for neural ODEs in generative modeling.



Training Adversarial Agents to Exploit Weaknesses in Deep Control Policies

An automated black box testing framework based on adversarial reinforcement learning is proposed, which is able to find weaknesses in both control policies that were not evident during online testing and therefore, demonstrate a significant benefit over manual testing methods.

Feature Purification: How Adversarial Training Performs Robust Deep Learning

A complexity lower bound is proved, showing that low complexity models such as linear classifiers, low-degree polynomials, or even the neural tangent kernel for this network, cannot defend against perturbations of this same radius, no matter what algorithms are used to train them.

Risk Averse Robust Adversarial Reinforcement Learning

It is shown through experiments that a risk-averse agent is better equipped to handle arisk-seeking adversary, and experiences substantially fewer crashes compared to agents trained without an adversary.

Provable defenses against adversarial examples via the convex outer adversarial polytope

A method to learn deep ReLU-based classifiers that are provably robust against norm-bounded adversarial perturbations, and it is shown that the dual problem to this linear program can be represented itself as a deep network similar to the backpropagation network, leading to very efficient optimization approaches that produce guaranteed bounds on the robust loss.

Adversarial Feature Training for Generalizable Robotic Visuomotor Control

It is demonstrated that by using adversarial training for domain transfer, it is possible to train visuomotor policies based on RL frameworks, and then transfer the acquired policy to other novel task domains, and the method is evaluated on two real robotic tasks, picking and pouring, to demonstrate its superiority.

Provably Robust Deep Learning via Adversarially Trained Smoothed Classifiers

It is demonstrated through extensive experimentation that this method consistently outperforms all existing provably $\ell-2$-robust classifiers by a significant margin on ImageNet and CIFAR-10, establishing the state-of-the-art for provable $\ell_ 2$-defenses.

Adversarial Machine Learning at Scale

This research applies adversarial training to ImageNet and finds that single-step attacks are the best for mounting black-box attacks, and resolution of a "label leaking" effect that causes adversarially trained models to perform better on adversarial examples than on clean examples.

Towards Deep Learning Models Resistant to Adversarial Attacks

This work studies the adversarial robustness of neural networks through the lens of robust optimization, and suggests the notion of security against a first-order adversary as a natural and broad security guarantee.

Disentangling Adversarial Robustness and Generalization

This work assumes an underlying, low-dimensional data manifold and shows that regular robustness and generalization are not necessarily contradicting goals, which implies that both robust and accurate models are possible.

Virtual Adversarial Training: A Regularization Method for Supervised and Semi-Supervised Learning

A new regularization method based on virtual adversarial loss: a new measure of local smoothness of the conditional label distribution given input that achieves state-of-the-art performance for semi-supervised learning tasks on SVHN and CIFAR-10.