# A policy iteration method for mean field games

@article{Cacace2021API, title={A policy iteration method for mean field games}, author={Simone Cacace and Fabio Camilli and Alessandro Goffi}, journal={ESAIM: Control, Optimisation and Calculus of Variations}, year={2021} }

The policy iteration method is a classical algorithm for solving optimal control problems. In this paper, we introduce a policy iteration method for Mean Field Games systems, and we study the convergence of this procedure to a solution of the problem. We also introduce suitable discretizations to numerically solve both stationary and evolutive problems. We show the convergence of the policy iteration method for the discrete problem and we study the performance of the proposed algorithm on some… Expand

#### 4 Citations

Numerical Methods for Mean Field Games and Mean Field Type Control

- Mathematics, Computer Science
- ArXiv
- 2021

Numerical schemes for forward-backward systems of partial differential equations (PDEs), optimization techniques for variational problems driven by a Kolmogorov-Fokker-Planck PDE, an approach based on a monotone operator viewpoint, and stochastic methods relying on machine learning tools are discussed. Expand

Policy iteration method for time-dependent Mean Field Games systems with non-separable Hamiltonians

- Computer Science, Mathematics
- ArXiv
- 2021

Two algorithms based on a policy iteration method to numerically solve time-dependent Mean Field Game systems of partial differential equations with non-separable Hamiltonians are introduced and it is proved that the convergence rates are linear. Expand

Rates of convergence for the policy iteration method for Mean Field Games systems

- Mathematics
- 2021

Convergence of the policy iteration method for discrete and continuous optimal control problems holds under general assumptions. Moreover, in some circumstances, it is also possible to show a… Expand

A Mean Field Games model for finite mixtures of Bernoulli and categorical distributions

- Mathematics
- 2020

Finite mixture models are an important tool in the statistical analysis of data, for example in data clustering. The optimal parameters of a mixture model are usually computed by maximizing the… Expand

#### References

SHOWING 1-10 OF 64 REFERENCES

On the Convergence of Policy Iteration in Stationary Dynamic Programming

- Mathematics, Computer Science
- Math. Oper. Res.
- 1979

It is shown to be equivalent to the Newton-Kantorovich iteration procedure applied to the functional equation of dynamic programming and this equivalence is used to obtain the rate of convergence and error bounds for the sequence of values generated by policy iteration. Expand

On the convergence of policy iteration for controlled diffusions

- Mathematics
- 1981

The convergence of an approximation scheme known as policy iteration has been demonstrated for controlled diffusions by Fleming, Puterman, and Bismut. In this paper, we show that this approximation… Expand

Mean Field Games: Convergence of a Finite Difference Method

- Mathematics, Computer Science
- SIAM J. Numer. Anal.
- 2013

Mean field type models describing the limiting behavior of stochastic differential games as the number of players tends to $+\infty$ have been recently introduced by Lasry and Lions. Numerical… Expand

Exponential Convergence and Stability of Howard's Policy Improvement Algorithm for Controlled Diffusions

- Mathematics, Computer Science
- SIAM J. Control. Optim.
- 2020

The first main contribution of this paper is to establish global rate of convergence for the policy improvement algorithm and a variant, called here the gradient iteration algorithm, and the proof of stability of the algorithms under perturbations to both the accuracy of the linear PDE solution and the accuracies of the maximization step. Expand

A Semi-Lagrangian scheme for a degenerate second order Mean Field Game system

- Mathematics
- 2014

In this paper we study a fully discrete Semi-Lagrangian approximation of a second order Mean Field Game system, which can be degenerate. We prove that the resulting scheme is well posed and, if the… Expand

On the turnpike property for mean field games

- Mathematics
- 2018

We consider the behavior of mean field games systems in the long horizon, under the assumption of monotonicity of the coupling term. Assuming that the Hamiltonian is globally Lipschitz and locally… Expand

Long time average of mean field games

- Mathematics, Computer Science
- Networks Heterog. Media
- 2012

It is shown that the model of mean field games system, rescaled in a suitable way, converges to a stationary ergodic mean field game and relies on energy estimates and the Hamiltonian structure of the system. Expand

Mean Field Games and Applications: Numerical Aspects

- Computer Science, Mathematics
- Lecture Notes in Mathematics
- 2020

In this survey, several aspects of a finite difference method used to approximate the previously mentioned system of PDEs are discussed, including convergence, variational aspects and algorithms for solving the resulting systems of nonlinear equations. Expand

Stable solutions in potential mean field game systems

- Mathematics
- 2016

We introduce the notion of stable solution in mean field game theory: they are locally isolated solutions of the mean field game system. We prove that such solutions exist in potential mean field… Expand

Proximal Methods for Stationary Mean Field Games with Local Couplings

- Computer Science, Mathematics
- SIAM J. Control. Optim.
- 2018

This work addresses the numerical approximation of mean field games with local couplings with power-like Hamiltonians by considering a stationary system and also a system involving density constraints modelin the model of Hamiltonians. Expand