Discounted Continuous Time Markov Decision Processes : the Convex Analytic Approach

  title={Discounted Continuous Time Markov Decision Processes : the Convex Analytic Approach},
  author={Alexei B. Piunovskiy},
The convex analytic approach which is dual, in some sense, to dynamic programming, is useful for the investigation of multicriteria control problems. It is well known for discrete time models, and the current paper presents similar results for the continuous time case. Namely, we define and study the space of occupation measures, and apply the abstract convex analysis to the study of constrained problems. Finally, we briefly consider a meaningful example on a controlled bicriteria Markovian… CONTINUE READING

From This Paper

Topics from this paper.


Publications referenced by this paper.
Showing 1-10 of 15 references

Constrained Markov Decision Processes, Chapman and Hall/CRC Press, London

  • E. Altman
  • 1999
Highly Influential
9 Excerpts

Optimal Control of Random Sequences in Problems with Constraints, Kluwer Academic Publishers, Dordrecht-Boston-London

  • A. B. Piunovskiy
  • 1997
Highly Influential
11 Excerpts

Constrained continuous-time Markov control processes with discounted criteria, Stoch

  • X. Guo, O.Hernandez-Lerma
  • Anal. Appl., 21, 379-400.
  • 2003

A controlled jump discounted model with constraints

  • A. B. Piunovskiy
  • Theory Probab . Appl .
  • 1998
2 Excerpts

A controlled jump discounted model with constraints, Theory Probab

  • A. B. Piunovskiy
  • Appl., 42, 51-72.
  • 1998
2 Excerpts

Controlled Queueing Systems, CRC Press, New York

  • M. Y. Kitaev
  • V.V.Rykov
  • 1995

On continuoustime discounted stochastic dynamic programming, Appl

  • H. C. Lai, K.Tanaka
  • Math. Optim., 23, 155-169.
  • 1991

Similar Papers

Loading similar papers…