• Corpus ID: 243848161

Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems

  title={Variational Automatic Curriculum Learning for Sparse-Reward Cooperative Multi-Agent Problems},
  author={Jiayu Chen and Yuanxin Zhang and Yuanfan Xu and Huimin Ma and Huazhong Yang and Jiaming Song and Yu Wang and Yi Wu},
We introduce a curriculum learning algorithm, Variational Automatic Curriculum Learning (VACL), for solving challenging goal-conditioned cooperative multiagent reinforcement learning problems. We motivate our paradigm through a variational perspective, where the learning objective can be decomposed into two terms: task learning on the current task distribution, and curriculum update to a new task distribution. Local optimization over the second term suggests that the curriculum should gradually… 
1 Citations
Self-Paced Multi-Agent Reinforcement Learning
Curriculum reinforcement learning (CRL) aims to speed up learning of a task by changing gradually the difficulty of the task from easy to hard through control of factors such as initial state or


Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
EPC is introduced, a curriculum learning paradigm that scales up Multi-Agent Reinforcement Learning (MARL) by progressively increasing the population of training agents in a stage-wise manner and uses an evolutionary approach to fix an objective misalignment issue throughout the curriculum.
Automated curriculum generation through setter-solver interactions
These results represent a substantial step towards applying automatic task curricula to learn complex, otherwise unlearnable goals, and to the authors' knowledge are the first to demonstrate automated curriculum generation for goal-conditioned agents in environments where the possible goals vary between episodes.
Unsupervised Curricula for Visual Meta-Reinforcement Learning
The algorithm allows for unsupervised meta- learning that both transfers to downstream tasks specified by hand-crafted reward functions and serves as pre-training for more efficient meta-learning of test task distributions.
BaRC: Backward Reachability Curriculum for Robotic Reinforcement Learning
The Backward Reachability Curriculum (BaRC) begins policy training from states that require a small number of actions to accomplish the task, and expands the initial state distribution backwards in a dynamically-consistent manner once the policy optimization algorithm demonstrates sufficient performance.
A Performance-Based Start State Curriculum Framework for Reinforcement Learning
This work proposes a unifying framework for performance-based start state curricula in RL, which allows to analyze and compare the performance influence of the two key components: performance measure estimation and a start selection policy.
Reverse Curriculum Generation for Reinforcement Learning
This work proposes a method to learn goal-oriented tasks without requiring any prior knowledge other than obtaining a single state in which the task is achieved, and generates a curriculum of start states that adapts to the agent's performance, leading to efficient training on goal- oriented tasks.
Teacher algorithms for curriculum learning of Deep RL in continuously parameterized environments
This work considers the problem of how a teacher algorithm can enable an unknown Deep Reinforcement Learning (DRL) student to become good at a skill over a wide range of diverse environments and presents a new algorithm modeling absolute learning progress with Gaussian mixture models (ALP-GMM).
From Few to More: Large-scale Dynamic Multiagent Curriculum Learning
A novel Dynamic Multiagent Curriculum Learning (DyMA-CL) to solve large-scale problems by starting from learning on a multiagent scenario with a small size and progressively increasing the number of agents, and proposes three transfer mechanisms across curricula to accelerate the learning process.
Cooperative Multi-Agent Learning: The State of the Art
This survey attempts to draw from multi-agent learning work in a spectrum of areas, including RL, evolutionary computation, game theory, complex systems, agent modeling, and robotics, and finds that this broad view leads to a division of the work into two categories.
Exploration via Hindsight Goal Generation
HGG is introduced, a novel algorithmic framework that generates valuable hindsight goals which are easy for an agent to achieve in the short term and are also potential for guiding the agent to reach the actual goal in the long term.