• Corpus ID: 238856732

Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games

  title={Shaping Large Population Agent Behaviors Through Entropy-Regularized Mean-Field Games},
  author={Yue Guan and Mi Zhou and Ali Pakniyat and Panagiotis Tsiotras},
  • Yue Guan, Mi Zhou, +1 author P. Tsiotras
  • Published 14 October 2021
  • Computer Science, Engineering
  • ArXiv
Mean-field games (MFG) were introduced to efficiently analyze approximate Nash equilibria in large population settings. In this work, we consider entropy-regularized meanfield games with a finite state-action space in a discrete time setting. We show that entropy regularization provides the necessary regularity conditions, that are lacking in the standard finite mean field games. Such regularity conditions enable us to design fixed-point iteration algorithms to find the unique meanfield… 

Figures from this paper


Markov-Nash equilibria in mean-field games with discounted cost
This paper demonstrates the existence of a mean-field equilibrium in the infinite-population limit, N → 1, and shows that the policy obtained from the mean- field equilibrium is approximately Markov-Nash when the number of agents N is sufficiently large.
Approximately Solving Mean Field Games via Entropy-Regularized Deep Reinforcement Learning
This paper shows that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration, and incorporates entropy-regularization and Boltzmann policies into theFixed point iteration.
Large population stochastic dynamic games: closed-loop McKean-Vlasov systems and the Nash certainty equivalence principle
The McKean-Vlasov NCE method presented in this paper has a close connection with the statistical physics of large particle systems: both identify a consistency relationship between the individual agent at the microscopic level and the mass of individuals at the macroscopic level.
Mean Field Multi-Agent Reinforcement Learning
Existing multi-agent reinforcement learning methods are limited typically to a small number of agents. When the agent number increases largely, the learning becomes intractable due to the curse of
Learning in Mean-Field Games
ADP techniques for design and adaptation (learning) of approximately optimal control laws for this model are introduced and a parameterization is proposed, based on an analysis of the mean-field PDE model for the game.
Mean Field Games: Numerical Methods
Numerical methods for the approximation of the stationary and evolutive versions of stochastic differential game models are proposed here and existence and uniqueness properties as well as bounds for the solutions of the discrete schemes are investigated.
Team optimal control of coupled subsystems with mean-field sharing
This work investigates team optimal control of stochastic subsystems that are weakly coupled in dynamics (through the mean-field of the system) and are arbitrary coupled in the cost and identifies an information state and uses that to obtain a dynamic programming decomposition.
Mean field games
Abstract.We survey here some recent studies concerning what we call mean-field models by analogy with Statistical Mechanics and Physics. More precisely, we present three examples of our mean-field
Mean-field models in swarm robotics: A survey.
The application of fluid approximations, in the form of mean-field models, to the design of control strategies in swarm robotics is surveyed, enabling new insights and provable guarantees on the dynamics of collective behaviors.
Mean Field Game-Theoretic Framework for Interference and Energy-Aware Control in 5G Ultra-Dense Networks
The mean field game can well satisfy the interference and energy-aware featured game requirements of 5G ultra-dense networks and is presented in D2D communications with interference and remaining energy dynamics.