Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes

  title={Simulation-Based Optimization Algorithms for Finite-Horizon Markov Decision Processes},
  author={Shalabh Bhatnagar and Mohammed Shahid Abdulla},
We develop four simulation-based algorithms for finite-horizon Markov decision processes. Two of these algorithms are developed for finite state and compact action spaces while the other two are for finite state and finite action spaces. Of the former two, one algorithm uses a linear parameterization for the policy, resulting in reduced memory complexity. Convergence analysis is briefly sketched and illustrative numerical experiments with the four algorithms are shown for a problem of flow… CONTINUE READING
4 Citations
39 References
Similar Papers


Publications citing this paper.
Showing 1-4 of 4 extracted citations


Publications referenced by this paper.
Showing 1-10 of 39 references

Multivariate stochastic approximation using a simultaneous perturbation gradient approximation

  • J. C. Spall
  • IEEE Transactions on Automatic Control
  • 1992
Highly Influential
6 Excerpts

Stochastic Approximation Methods for Constrained and Unconstrained Systems

  • H. Kushner, D. Clark
  • 1978
Highly Influential
6 Excerpts

Handbook of Markov Decision Processes: Methods and Applications

  • B. Van Roy
  • 2001
Highly Influential
4 Excerpts

Similar Papers

Loading similar papers…