Corpus ID: 104291994

Active Domain Randomization

@inproceedings{Mehta2019ActiveDR,
  title={Active Domain Randomization},
  author={Bhairav Mehta and Manfred Diaz and Florian Golemo and Christopher Joseph Pal and Liam Paull},
  booktitle={CoRL},
  year={2019}
}
Domain randomization is a popular technique for improving domain transfer, often used in a zero-shot setting when the target domain is unknown or cannot easily be used for training. In this work, we empirically examine the effects of domain randomization on agent generalization. Our experiments show that domain randomization may lead to suboptimal, high-variance policies, which we attribute to the uniform sampling of environment parameters. We propose Active Domain Randomization, a novel… Expand
Robust Domain Randomization for Reinforcement Learning
TLDR
This work formalizes the domain randomization problem, and shows that minimizing the policy's Lipschitz constant with respect to the randomization parameters leads to low variance in the learned policies. Expand
Bayesian Domain Randomization for Sim-to-Real Transfer
TLDR
Bayesian Domain Randomization (BayRn), a black box sim-to-real algorithm that solves tasks efficiently by adapting the domain parameter distribution during learning by sampling the real-world target domain, is proposed. Expand
Effects of Task Distributions on Policy Optimization
Domain randomization is a popular technique for improving domain transfer, often used in a zero-shot setting when the target domain is unknown or cannot easily be used for training. In this work, weExpand
Learning Domain Randomization Distributions for Training Robust Locomotion Policies
TLDR
It is shown how adapting the DR distribution while training context-conditioned policies results in improvements on jump-start and asymptotic performance when transferring a learned policy to the target environment. Expand
Robust Visual Domain Randomization for Reinforcement Learning.
Producing agents that can generalize to a wide range of visually different environments is a significant challenge in reinforcement learning. One method for overcoming this issue is visual domainExpand
Bayesian Optimization for Learning from Randomized Simulations
Physics simulators offer a powerful way to develop and test policies in robotics. Promising progress in learning difficult robotic control tasks has recently been made using reinforcement learning,Expand
A Simple Feature Augmentation for Domain Generalization
The topical domain generalization (DG) problem asks trained models to perform well on an unseen target domain with different data statistics from the source training domains. In computer vision, dataExpand
Not Only Domain Randomization: Universal Policy with Embedding System Identification
  • Zihan Ding
  • Computer Science
  • ArXiv
  • 2021
TLDR
This paper presents Universal Policy with Embedding System Identification (UPESI) as an implicit system identification (SI) approach with universal policies (UPs), as a learning-based control method to execute optimal actions adaptively in environments with various dynamic properties. Expand
Curriculum in Gradient-Based Meta-Reinforcement Learning
TLDR
Meta Active Domain Randomization ( meta-ADR), which learns a curriculum of tasks for gradient-based meta-RL in a similar as ADR does for sim2real transfer, is proposed and it is found that this approach induces more stable policies on a variety of simulated locomotion and navigation tasks. Expand
HARD-EXPLORATION PROBLEMS
We introduce Adaptive Procedural Task Generation (APT-Gen), an approach to progressively generate a sequence of tasks as curricula to facilitate reinforcement learning in hard-exploration problems.Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 40 REFERENCES
Generalization and Regularization in DQN
TLDR
Despite regularization being largely underutilized in deep RL, it is shown that it can, in fact, help DQN learn more general features and can then be reused and fine-tuned on similar tasks, considerably improving the sample efficiency of D QN. Expand
VADRA: Visual Adversarial Domain Randomization and Augmentation
TLDR
This work proposes a novel algorithm which closes the loop between the synthetic generative model and the learner in an adversarial fashion and evaluates its method on diverse vision tasks using state-of-the-art simulators for public datasets like CLEVR, Syn2Real, and VIRAT, to demonstrate that a learner trained using adversarial data generation performs better than using a random data generation strategy. Expand
Domain randomization for transferring deep neural networks from simulation to the real world
TLDR
This paper explores domain randomization, a simple technique for training models on simulated images that transfer to real images by randomizing rendering in the simulator, and achieves the first successful transfer of a deep neural network trained only on simulated RGB images to the real world for the purpose of robotic control. Expand
Assessing Generalization in Deep Reinforcement Learning
TLDR
The key finding is that `vanilla' deep RL algorithms generalize better than specialized schemes that were proposed specifically to tackle generalization. Expand
Learning Self-Imitating Diverse Policies
TLDR
A self-imitation learning algorithm that exploits and explores well in the sparse and episodic reward settings, and can be reduced into a policy-gradient algorithm with shaped rewards learned from experience replays with Stein variational policy gradient descent with Jensen-Shannon divergence. Expand
Composable Deep Reinforcement Learning for Robotic Manipulation
TLDR
This paper shows that policies learned with soft Q-learning can be composed to create new policies, and that the optimality of the resulting policy can be bounded in terms of the divergence between the composed policies. Expand
Closing the Sim-to-Real Loop: Adapting Simulation Randomization with Real World Experience
TLDR
This work adapts the simulation parameter distribution using a few real world roll-outs interleaved with policy training to improve the policy transfer by matching the policy behavior in simulation and the real world. Expand
Diversity is All You Need: Learning Skills without a Reward Function
TLDR
The proposed DIAYN ("Diversity is All You Need"), a method for learning useful skills without a reward function, learns skills by maximizing an information theoretic objective using a maximum entropy policy. Expand
Quantifying Generalization in Reinforcement Learning
TLDR
It is shown that deeper convolutional architectures improve generalization, as do methods traditionally found in supervised learning, including L2 regularization, dropout, data augmentation and batch normalization. Expand
Learning To Simulate
TLDR
This work proposes a reinforcement learning-based method for automatically adjusting the parameters of any (non-differentiable) simulator, thereby controlling the distribution of synthesized data in order to maximize the accuracy of a model trained on that data. Expand
...
1
2
3
4
...