# Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games

@article{Guan2021ShapingLP, title={Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games}, author={Yue Guan and Mi Zhou and Ali Pakniyat and Panagiotis Tsiotras}, journal={ArXiv}, year={2021}, volume={abs/2110.07469} }

Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized meanfield games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique meanfield…

## References

SHOWING 1-10 OF 28 REFERENCES

Markov-Nash equilibria in mean-field games with discounted cost

- Mathematics, Computer Science2017 American Control Conference (ACC)
- 2017

This paper demonstrates the existence of a mean-field equilibrium in the infinite-population limit, N → 1, and shows that the policy obtained from the mean- field equilibrium is approximately Markov-Nash when the number of agents N is sufficiently large.

Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning

- Computer ScienceAISTATS
- 2021

This paper shows that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration, and incorporates entropy-regularization and Boltzmann policies into theFixed point iteration.

Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle

- Mathematics, Computer ScienceCommun. Inf. Syst.
- 2006

The McKean-Vlasov NCE method presented in this paper has a close connection with the statistical physics of large particle systems: both identify a consistency relationship between the individual agent at the microscopic level and the mass of individuals at the macroscopic level.

Mean Field Multi-Agent Reinforcement Learning

- Computer ScienceICML
- 2018

Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of…

Learning in Mean-Field Games

- Mathematics, Computer ScienceIEEE Transactions on Automatic Control
- 2014

ADP techniques for design and adaptation (learning) of approximately optimal control laws for this model are introduced and a parameterization is proposed, based on an analysis of the mean-field PDE model for the game.

Mean Field Games: Numerical Methods

- Mathematics, Computer ScienceSIAM J. Numer. Anal.
- 2010

Numerical methods for the approximation of the stationary and evolutive versions of stochastic differential game models are proposed here and existence and uniqueness properties as well as bounds for the solutions of the discrete schemes are investigated.

Team optimal control of coupled subsystems with mean-field sharing

- Computer Science, Mathematics53rd IEEE Conference on Decision and Control
- 2014

This work investigates team optimal control of stochastic subsystems that are weakly coupled in dynamics (through the mean-field of the system) and are arbitrary coupled in the cost and identifies an information state and uses that to obtain a dynamic programming decomposition.

Mean field games

- Mathematics
- 2007

Abstract.We survey here some recent studies concerning what we call mean-field models by analogy with Statistical Mechanics and Physics. More precisely, we present three examples of our mean-field…

Mean-field models in swarm robotics: A survey.

- Medicine, Computer ScienceBioinspiration & biomimetics
- 2019

The application of fluid approximations, in the form of mean-field models, to the design of control strategies in swarm robotics is surveyed, enabling new insights and provable guarantees on the dynamics of collective behaviors.

Mean Field Game-Theoretic Framework for Interference and Energy-Aware Control in 5G Ultra-Dense Networks

- Computer ScienceIEEE Wireless Communications
- 2018

The mean field game can well satisfy the interference and energy-aware featured game requirements of 5G ultra-dense networks and is presented in D2D communications with interference and remaining energy dynamics.