Corpus ID: 211021045

Preventing Imitation Learning with Adversarial Policy Ensembles

  title={Preventing Imitation Learning with Adversarial Policy Ensembles},
  author={Albert Zhan and Stas Tiomkin and P. Abbeel},
Imitation learning can reproduce policies by observing experts, which poses a problem regarding policy privacy. Policies, such as human, or policies on deployed robots, can all be cloned without consent from the owners. How can we protect against external observers cloning our proprietary policies? To answer this question we introduce a new reinforcement learning framework, where we train an ensemble of near-optimal policies, whose demonstrations are guaranteed to be useless for an external… Expand
Adversarial jamming attacks and defense strategies via adaptive deep reinforcement learning
This paper considers a victim user that performs DRL-based dynamic channel access, and an attacker that executes DRLbased jamming attacks to disrupt the victim, and proposes three defense strategies, namely diversified defense with proportional-integral-derivative (PID) control, diversifieddefense with an imitation attacker, and defense via orthogonal policies. Expand
Learning-based attacks in Cyber-Physical Systems: Exploration, Detection, and Control Cost trade-offs
The problem of learning-based attacks in linear systems, where the communication channel between the controller and the plant can be hijacked by a malicious attacker, is studied and a probabilistic lower bound on the time that must be spent by the attacker learning the system is shown. Expand


Adversarial Policies: Attacking Deep Reinforcement Learning
The existence of adversarial policies in zero-sum games between simulated humanoid robots with proprioceptive observations, against state-of-the-art victims trained via self-play to be robust to opponents is demonstrated. Expand
Generative Adversarial Imitation Learning
A new general framework for directly extracting a policy from data, as if it were obtained by reinforcement learning following inverse reinforcement learning, is proposed and a certain instantiation of this framework draws an analogy between imitation learning and generative adversarial networks. Expand
Iterative Noise Injection for Scalable Imitation Learning
An improved bound on the loss due to the covariate shift is proved, and an algorithm that leverages the analysis to estimate the level of -greedy noise to inject is introduced that achieves a better performance than DAgger with 75% fewer demonstrations. Expand
OptionGAN: Learning Joint Reward-Policy Options using Generative Adversarial Inverse Reinforcement Learning
This work uses adversarial methods to learn joint reward-policy options using only observed expert states and shows significant performance increases in one-shot transfer learning. Expand
Variational Discriminator Bottleneck: Improving Imitation Learning, Inverse RL, and GANs by Constraining Information Flow
This work proposes a simple and general technique to constrain information flow in the discriminator by means of an information bottleneck, and demonstrates that the proposed variational discriminator bottleneck (VDB) leads to significant improvements across three distinct application areas for adversarial learning algorithms. Expand
Policy Poisoning in Batch Reinforcement Learning and Control
This work presents a unified framework for solving batch policy poisoning attacks, and instantiate the attack on two standard victims: tabular certainty equivalence learner in reinforcement learning and linear quadratic regulator in control. Expand
Vulnerability of Deep Reinforcement Learning to Policy Induction Attacks
This work establishes that reinforcement learning techniques based on Deep Q-Networks are also vulnerable to adversarial input perturbations, and presents a novel class of attacks based on this vulnerability that enable policy manipulation and induction in the learning process of DQNs. Expand
Privacy-preserving Q-Learning with Functional Noise in Continuous State Spaces
We consider differentially private algorithms for reinforcement learning in continuous spaces, such that neighboring reward functions are indistinguishable. This protects the reward information fromExpand
Tactics of Adversarial Attack on Deep Reinforcement Learning Agents
A novel method to determine when an adversarial example should be crafted and applied is proposed, namely the strategically-timed attack and the enchanting attack, which are introduced to attack agents trained by deep reinforcement learning algorithms using adversarial examples. Expand
Apprenticeship learning via inverse reinforcement learning
This work thinks of the expert as trying to maximize a reward function that is expressible as a linear combination of known features, and gives an algorithm for learning the task demonstrated by the expert, based on using "inverse reinforcement learning" to try to recover the unknown reward function. Expand