# Preventing Imitation Learning with Adversarial Policy Ensembles

@article{Zhan2020PreventingIL, title={Preventing Imitation Learning with Adversarial Policy Ensembles}, author={Albert Zhan and Stas Tiomkin and P. Abbeel}, journal={ArXiv}, year={2020}, volume={abs/2002.01059} }

Imitation learning can reproduce policies by observing experts, which poses a problem regarding policy privacy. Policies, such as human, or policies on deployed robots, can all be cloned without consent from the owners. How can we protect against external observers cloning our proprietary policies? To answer this question we introduce a new reinforcement learning framework, where we train an ensemble of near-optimal policies, whose demonstrations are guaranteed to be useless for an external… Expand

#### 2 Citations

Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning

- Computer Science, Engineering
- ArXiv
- 2020

This paper considers a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim, and proposes three defense strategies, namely diversified defense with proportional-integral-derivative (PID) control, diversifieddefense with an imitation attacker, and defense via orthogonal policies. Expand

Learning-based attacks in Cyber-Physical Systems: Exploration, Detection, and Control Cost trade-offs

- Computer Science, Engineering
- L4DC
- 2021

The problem of learning-based attacks in linear systems, where the communication channel between the controller and the plant can be hijacked by a malicious attacker, is studied and a probabilistic lower bound on the time that must be spent by the attacker learning the system is shown. Expand

#### References

SHOWING 1-10 OF 49 REFERENCES

Adversarial Policies: Attacking Deep Reinforcement Learning

- Computer Science, Mathematics
- ICLR
- 2020

The existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents is demonstrated. Expand

Generative Adversarial Imitation Learning

- Computer Science, Mathematics
- NIPS
- 2016

A new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning, is proposed and a certain instantiation of this framework draws an analogy between imitation learning and generative adversarial networks. Expand

Iterative Noise Injection for Scalable Imitation Learning

- Computer Science
- ArXiv
- 2017

An improved bound on the loss due to the covariate shift is proved, and an algorithm that leverages the analysis to estimate the level of -greedy noise to inject is introduced that achieves a better performance than DAgger with 75% fewer demonstrations. Expand

OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning

- Computer Science, Mathematics
- AAAI
- 2018

This work uses adversarial methods to learn joint reward-policy options using only observed expert states and shows significant performance increases in one-shot transfer learning. Expand

Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow

- Computer Science, Mathematics
- ICLR
- 2019

This work proposes a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck, and demonstrates that the proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms. Expand

Policy Poisoning in Batch Reinforcement Learning and Control

- Computer Science, Mathematics
- NeurIPS
- 2019

This work presents a unified framework for solving batch policy poisoning attacks, and instantiate the attack on two standard victims: tabular certainty equivalence learner in reinforcement learning and linear quadratic regulator in control. Expand

Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks

- Computer Science, Mathematics
- MLDM
- 2017

This work establishes that reinforcement learning techniques based on Deep Q-Networks are also vulnerable to adversarial input perturbations, and presents a novel class of attacks based on this vulnerability that enable policy manipulation and induction in the learning process of DQNs. Expand

Privacy-preserving Q-Learning with Functional Noise in Continuous State Spaces

- Mathematics
- 2019

We consider differentially private algorithms for reinforcement learning in continuous spaces, such that neighboring reward functions are indistinguishable. This protects the reward information from… Expand

Tactics of Adversarial Attack on Deep Reinforcement Learning Agents

- Computer Science, Mathematics
- IJCAI
- 2017

A novel method to determine when an adversarial example should be crafted and applied is proposed, namely the strategically-timed attack and the enchanting attack, which are introduced to attack agents trained by deep reinforcement learning algorithms using adversarial examples. Expand

Apprenticeship learning via inverse reinforcement learning

- Computer Science
- ICML
- 2004

This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function. Expand