Heterogeneous Multi-Robot Reinforcement Learning

@article{Bettini2023HeterogeneousMR,
  title={Heterogeneous Multi-Robot Reinforcement Learning},
  author={Matteo Bettini and Ajay Shankar and Amanda Prorok},
  journal={ArXiv},
  year={2023},
  volume={abs/2301.07137}
}
Cooperative multi-robot tasks can benefit from heterogeneity in the robots’ physical and behavioral traits. In spite of this, traditional Multi-Agent Reinforcement Learning (MARL) frameworks lack the ability to explicitly accommodate policy heterogeneity, and typically constrain agents to share neural network parameters. This enforced homogeneity limits application in cases where the tasks benefit from heterogeneous behaviors. In this paper, we crystallize the role of heterogeneity in MARL… 
View PDF on arXiv

Figures and Tables from this paper

References

SHOWING 1-10 OF 61 REFERENCES

Heterogeneous Multi-Agent Reinforcement Learning for Unknown Environment Mapping

This work presents an actor-critic algorithm that allows a team of heterogeneous agents to learn decentralized control policies for covering an unknown environment, and develops a simulation environment that includes real-world environmental factors such as turbulence, delayed communication, and agent loss to train teams of agents.

Multi-Agent Actor-Critic for Mixed Cooperative-Competitive Environments

An adaptation of actor-critic methods that considers action policies of other agents and is able to successfully learn policies that require complex multi-agent coordination is presented.

ROMA: Multi-Agent Reinforcement Learning with Emergent Roles

Experiments show that the proposed role-oriented MARL framework (ROMA) can learn specialized, dynamic, and identifiable roles, which help the method push forward the state of the art on the StarCraft II micromanagement benchmark.

Learning to Communicate with Deep Multi-Agent Reinforcement Learning

By embracing deep neural networks, this work is able to demonstrate end-to-end learning of protocols in complex environments inspired by communication riddles and multi-agent computer vision problems with partial observability.

Revisiting Parameter Sharing in Multi-Agent Deep Reinforcement Learning

It is shown that increasing centralization during learning arbitrarily mitigates the slowing of convergence due to nonstationarity and gives a formal proof of a set of methods that allow parameter sharing to serve in environments with heterogeneous agents.

Natural Emergence of Heterogeneous Strategies in Artificially Intelligent Competitive Teams

This work develops a competitive multi agent environment called FortAttack in which two teams compete against each other, and corroborates that modeling agents with Graph Neural Networks and training them with Reinforcement Learning leads to the evolution of increasingly complex strategies for each team.

Cooperative Multi-agent Control Using Deep Reinforcement Learning

It is shown that policy gradient methods tend to outperform both temporal-difference and actor-critic methods and that curriculum learning is vital to scaling reinforcement learning algorithms in complex multi-agent domains.

The StarCraft Multi-Agent Challenge

The StarCraft Multi-Agent Challenge (SMAC), based on the popular real-time strategy game StarCraft II, is proposed as a benchmark problem and an open-source deep multi-agent RL learning framework including state-of-the-art algorithms is opened.

Counterfactual Multi-Agent Policy Gradients

A new multi-agent actor-critic method called counterfactual multi- agent (COMA) policy gradients that uses a centralised critic to estimate the Q-function and decentralised actors to optimise the agents' policies.

Learning Efficient Diverse Communication for Cooperative Heterogeneous Teaming

This work proposes Heterogeneous Policy Networks (HetNet) to learn efficient and diverse communication models for coordinating cooperative heterogeneous teams and shows that HetNet not only facilitates learning heterogeneous collaborative policies per existing agent-class but also enables end-to-end training for learning highly efficient binarized messaging.
...