• Corpus ID: 224897359

Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning

  title={Randomized Entity-wise Factorization for Multi-Agent Reinforcement Learning},
  author={Shariq Iqbal and C. S. D. Witt and Bei Peng and Wendelin Bohmer and Shimon Whiteson and Fei Sha},
Real world multi-agent tasks often involve varying types and quantities of agents and non-agent entities; however, agents within these tasks rarely need to consider all others at all times in order to act effectively. Factored value function approaches have historically leveraged such independences to improve learning efficiency, but these approaches typically rely on domain knowledge to select fixed subsets of state features to include in each factor. We propose to utilize value function… 

Figures and Tables from this paper

On the Use and Misuse of Absorbing States in Multi-agent Reinforcement Learning
A novel architecture is presented for an existing state-of-the-art MARL algorithm which uses attention instead of a fully connected layer with absorbing states and significantly outperforms the standard architecture on tasks in which agents are created or destroyed within episodes as well as standard multi-agent coordination.
Celebrating Diversity in Shared Multi-Agent Reinforcement Learning
This paper proposes an information-theoretical regularization to maximize the mutual information between agents’ identities and their trajectories, encouraging extensive exploration and diverse individualized behaviors in shared multi-agent reinforcement learning.
Learning Meta Representations for Agents in Multi-Agent Reinforcement Learning
This work proposes Meta Representations for Agents (MRA) that explicitly models the game-common and game-specific strategic knowledge and proves that as an approximation to a constrained mutual information maximization objective, the learned policies can reach Nash Equilibrium in every evaluation MG under the assumption of Lipschitz game on a sufficiently large latent space.
Meta-CPR: Generalize to Unseen Large Number of Agents with Communication Pattern Recognition Module
A meta reinforcement learning ( meta-RL) framework is proposed to tackle the problem of effective communication mechanism among agents in reinforcement learning and employs a meta-learned Communication Pattern Recognition module to identify communication behavior and extract information that facilitates the training process.
Snowflake: Scaling GNNs to High-Dimensional Continuous Control via Parameter Freezing
SNOWFLAKE is introduced, a GNN training method for high-dimensional continuous control that freezes parameters in parts of the network that suffer from overfitting, and significantly boosts the performance of GNNs for locomotion control on large agents, now matching theperformance of MLPs, and with superior transfer properties.


Deep Reinforcement Learning with Double Q-Learning
This paper proposes a specific adaptation to the DQN algorithm and shows that the resulting algorithm not only reduces the observed overestimations, as hypothesized, but that this also leads to much better performance on several games.
RODE: Learning Roles to Decompose Multi-Agent Tasks
This work proposes to first decompose joint action spaces into restricted role action spaces by clustering actions according to their effects on the environment and other agents by integrating information about action effects into the role policies to boost learning efficiency and policy generalization.
"Other-Play" for Zero-Shot Coordination
This work introduces a novel learning algorithm called other-play (OP), that enhances self-play by looking for more robust strategies, exploiting the presence of known symmetries in the underlying problem.
Action Semantics Network: Considering the Effects of Actions in Multiagent Systems
A novel network architecture, named Action Semantics Network (ASN), is proposed that characterizes different actions' influence on other agents using neural networks based on the action semantics between them and can be easily combined with existing deep reinforcement learning algorithms to boost their performance.
Deep Coordination Graphs
It is shown that DCG can solve challenging predator-prey tasks that are vulnerable to the relative overgeneralization pathology and in which all other known value factorization approaches fail.
Deep Multi-Agent Reinforcement Learning in Starcraft II
  • 2020
Emergent Tool Use From Multi-Agent Autocurricula
This work finds clear evidence of six emergent phases in agent strategy in the authors' environment, each of which creates a new pressure for the opposing team to adapt, and compares hide-and-seek agents to both intrinsic motivation and random initialization baselines in a suite of domain-specific intelligence tests.
Evolutionary Population Curriculum for Scaling Multi-Agent Reinforcement Learning
EPC is introduced, a curriculum learning paradigm that scales up Multi-Agent Reinforcement Learning (MARL) by progressively increasing the population of training agents in a stage-wise manner and uses an evolutionary approach to fix an objective misalignment issue throughout the curriculum.
From Few to More: Large-scale Dynamic Multiagent Curriculum Learning
A novel Dynamic Multiagent Curriculum Learning (DyMA-CL) to solve large-scale problems by starting from learning on a multiagent scenario with a small size and progressively increasing the number of agents, and proposes three transfer mechanisms across curricula to accelerate the learning process.