A policy iteration method for mean field games

  title={A policy iteration method for mean field games},
  author={Simone Cacace and Fabio Camilli and Alessandro Goffi},
  journal={ESAIM: Control, Optimisation and Calculus of Variations},
The policy iteration method is a classical algorithm for solving optimal control problems. In this paper, we introduce a policy iteration method for Mean Field Games systems, and we study the convergence of this procedure to a solution of the problem. We also introduce suitable discretizations to numerically solve both stationary and evolutive problems. We show the convergence of the policy iteration method for the discrete problem and we study the performance of the proposed algorithm on some… Expand

Figures and Tables from this paper

Numerical Methods for Mean Field Games and Mean Field Type Control
Numerical schemes for forward-backward systems of partial differential equations (PDEs), optimization techniques for variational problems driven by a Kolmogorov-Fokker-Planck PDE, an approach based on a monotone operator viewpoint, and stochastic methods relying on machine learning tools are discussed. Expand
Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians
Two algorithms based on a policy iteration method to numerically solve time-dependent Mean Field Game systems of partial differential equations with non-separable Hamiltonians are introduced and it is proved that the convergence rates are linear. Expand
Rates of convergence for the policy iteration method for Mean Field Games systems
Convergence of the policy iteration method for discrete and continuous optimal control problems holds under general assumptions. Moreover, in some circumstances, it is also possible to show aExpand
A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions
Finite mixture models are an important tool in the statistical analysis of data, for example in data clustering. The optimal parameters of a mixture model are usually computed by maximizing theExpand


On the Convergence of Policy Iteration in Stationary Dynamic Programming
It is shown to be equivalent to the Newton-Kantorovich iteration procedure applied to the functional equation of dynamic programming and this equivalence is used to obtain the rate of convergence and error bounds for the sequence of values generated by policy iteration. Expand
On the convergence of policy iteration for controlled diffusions
The convergence of an approximation scheme known as policy iteration has been demonstrated for controlled diffusions by Fleming, Puterman, and Bismut. In this paper, we show that this approximationExpand
Mean Field Games: Convergence of a Finite Difference Method
Mean field type models describing the limiting behavior of stochastic differential games as the number of players tends to $+\infty$ have been recently introduced by Lasry and Lions. NumericalExpand
Exponential Convergence and Stability of Howard's Policy Improvement Algorithm for Controlled Diffusions
The first main contribution of this paper is to establish global rate of convergence for the policy improvement algorithm and a variant, called here the gradient iteration algorithm, and the proof of stability of the algorithms under perturbations to both the accuracy of the linear PDE solution and the accuracies of the maximization step. Expand
A Semi-Lagrangian scheme for a degenerate second order Mean Field Game system
In this paper we study a fully discrete Semi-Lagrangian approximation of a second order Mean Field Game system, which can be degenerate. We prove that the resulting scheme is well posed and, if theExpand
On the turnpike property for mean field games
We consider the behavior of mean field games systems in the long horizon, under the assumption of monotonicity of the coupling term. Assuming that the Hamiltonian is globally Lipschitz and locallyExpand
Long time average of mean field games
It is shown that the model of mean field games system, rescaled in a suitable way, converges to a stationary ergodic mean field game and relies on energy estimates and the Hamiltonian structure of the system. Expand
Mean Field Games and Applications: Numerical Aspects
In this survey, several aspects of a finite difference method used to approximate the previously mentioned system of PDEs are discussed, including convergence, variational aspects and algorithms for solving the resulting systems of nonlinear equations. Expand
Stable solutions in potential mean field game systems
We introduce the notion of stable solution in mean field game theory: they are locally isolated solutions of the mean field game system. We prove that such solutions exist in potential mean fieldExpand
Proximal Methods for Stationary Mean Field Games with Local Couplings
This work addresses the numerical approximation of mean field games with local couplings with power-like Hamiltonians by considering a stationary system and also a system involving density constraints modelin the model of Hamiltonians. Expand