• Corpus ID: 219980275

Automatic Data Augmentation for Generalization in Deep Reinforcement Learning

@article{Raileanu2020AutomaticDA,
  title={Automatic Data Augmentation for Generalization in Deep Reinforcement Learning},
  author={Roberta Raileanu and Maxwell Goldstein and Denis Yarats and Ilya Kostrikov and Rob Fergus},
  journal={ArXiv},
  year={2020},
  volume={abs/2006.12862}
}
Deep reinforcement learning (RL) agents often fail to generalize to unseen scenarios, even when they are trained on many instances of semantically similar environments. Data augmentation has recently been shown to improve the sample efficiency and generalization of RL agents. However, different tasks tend to benefit from different kinds of data augmentation. In this paper, we compare three approaches for automatically finding an appropriate augmentation. These are combined with two novel… 
Improving Generalization in Reinforcement Learning with Mixture Regularization
TLDR
This work introduces a simple approach, named mixreg, which trains agents on a mixture of observations from different training environments and imposes linearity constraints on the observation interpolations and the supervision and increases the data diversity more effectively and helps learn smoother policies.
Contrastive Behavioral Similarity Embeddings for Generalization in Reinforcement Learning
TLDR
A theoretically motivated policy similarity metric (PSM) for measuring behavioral similarity between states is introduced and it is demonstrated that PSEs improve generalization on diverse benchmarks, including LQR with spurious correlations, a jumping task from pixels, and Distracting DM Control Suite.
Advances in Electron Microscopy with Deep Learning
TLDR
Thesis Structure, Connections, and Partial Scanning Transmission Electron Microscopy with Deep Learning: A Review of the Building Blocks of Deep Learning.
Deep learning in electron microscopy
  • Jeffrey M. Ede
  • Computer Science, Physics
    Mach. Learn. Sci. Technol.
  • 2021
TLDR
This review paper offers a practical perspective aimed at developers with limited familiarity of deep learning in electron microscopy that discusses hardware and software needed to get started with deep learning and interface with electron microscopes.
Supplementary for Neuro-algorithmic Policies Enable Fast Combinatorial Generalization
Since the environments become deterministic for a fixed random seed, we first unrolled their dynamics for each level. After obtaining the underlying grid structure and entities, we labeled them with
Time Matters in Using Data Augmentation for Vision-based Deep Reinforcement Learning
TLDR
According to experiments on Open AI Procgen Benchmark, if the regularization imposed by augmentation is helpful only in testing, it is better to procrastinate the augmentation after training than to use it during training in terms of sample and computation complexity.
Deep Reinforcement Learning at the Edge of the Statistical Precipice
TLDR
This paper argues that reliable evaluation in the few-run deep RL regime cannot ignore the uncertainty in results without running the risk of slowing down progress in the field, and advocates for reporting interval estimates of aggregate performance and proposing performance profiles to account for the variability in results.
Procedural Generalization by Planning with Self-Supervised World Models
TLDR
Overall, this work suggests that building generalizable agents requires moving beyond the single-task, model-free paradigm and towards self-supervised model-based agents that are trained in rich, procedural, multi-task environments.
Measuring Visual Generalization in Continuous Control from Pixels
TLDR
The empirical analysis shows that current methods struggle to generalize across a diverse set of visual changes, and it is found that data augmentation techniques outperform self-supervised learning approaches and that more significant image transformations provide better visual generalization.
Automated Reinforcement Learning (AutoRL): A Survey and Open Problems
TLDR
This survey seeks to unify the field of AutoRL, provide a common taxonomy, discuss each area in detail and pose open problems which would be of interest to researchers going forward.
...
1
2
3
4
...

References

SHOWING 1-10 OF 62 REFERENCES
Quantifying Generalization in Reinforcement Learning
TLDR
It is shown that deeper convolutional architectures improve generalization, as do methods traditionally found in supervised learning, including L2 regularization, dropout, data augmentation and batch normalization.
Generalization and Regularization in DQN
TLDR
Despite regularization being largely underutilized in deep RL, it is shown that it can, in fact, help DQN learn more general features and can then be reused and fine-tuned on similar tasks, considerably improving the sample efficiency of D QN.
Network Randomization: A Simple Technique for Generalization in Deep Reinforcement Learning
TLDR
A simple technique to improve a generalization ability of deep RL agents by introducing a randomized (convolutional) neural network that randomly perturbs input observations, which enables trained agents to adapt to new domains by learning robust features invariant across varied and randomized environments.
Automatic Data Augmentation by Learning the Deterministic Policy
TLDR
A novel learning-based augmentation method termed as DeepAugNet, which formulates the final augmented data as a collection of several sequentially augmented subsets, which can achieve a significant improvement in small-scale datasets, and a comparable performance in large- scale datasets.
Generalization in Reinforcement Learning with Selective Noise Injection and Information Bottleneck
TLDR
This work proposes Selective Noise Injection (SNI), which maintains the regularizing effect the injected noise has, while mitigating the adverse effects it has on the gradient quality, and demonstrates that the Information Bottleneck is a particularly well suited regularization technique for RL as it is effective in the low-data regime encountered early on in training RL agents.
Illuminating Generalization in Deep Reinforcement Learning through Procedural Level Generation
TLDR
It is shown that for some games procedural level generation enables generalization to new levels within the same distribution and it is possible to achieve better performance with less data by manipulating the difficulty of the levels in response to the performance of the agent.
Learning to reinforcement learn
TLDR
This work introduces a novel approach to deep meta-reinforcement learning, which is a system that is trained using one RL algorithm, but whose recurrent dynamics implement a second, quite separate RL procedure.
Model-Agnostic Meta-Learning for Fast Adaptation of Deep Networks
We propose an algorithm for meta-learning that is model-agnostic, in the sense that it is compatible with any model trained with gradient descent and applicable to a variety of different learning
IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures
TLDR
A new distributed agent IMPALA (Importance Weighted Actor-Learner Architecture) is developed that not only uses resources more efficiently in single-machine training but also scales to thousands of machines without sacrificing data efficiency or resource utilisation.
Observational Overfitting in Reinforcement Learning
TLDR
This work provides a general framework for analyzing the case where the agent may mistakenly correlate reward with certain spurious features from the observations generated by the Markov Decision Process, and designs multiple synthetic benchmarks from only modifying the observation space of an MDP.
...
1
2
3
4
5
...