The Gauss-Seidel numerical procedure for Markov stochastic games

@article{Kushner2004TheGN,
  title={The Gauss-Seidel numerical procedure for Markov stochastic games},
  author={Harold J. Kushner},
  journal={IEEE Transactions on Automatic Control},
  year={2004},
  volume={49},
  pages={1779-1784}
}
Consider the problem of value iteration for solving Markov stochastic games. One simply iterates backward, via a Jacobi-like procedure. The convergence of the Gauss-Seidel form of this procedure is shown for both the discounted and ergodic cost problems, under appropriate conditions, with extensions to problems where one stops when a boundary is hit or if any one of the players chooses to stop, with associated costs. Generally, the Gauss-Seidel procedure accelerates convergence.