• Mathematics, Computer Science
  • Published in ISAIM 2008

Solving Structured Continuous-Time Markov Decision Processes

  title={Solving Structured Continuous-Time Markov Decision Processes},
  author={Kin Fai Kan and Christian R. Shelton},
We present an approach to solving structured continuous-time Markov decision processes. We approximate the the optimal value function by a compact linear form, resulting in a linear program. The main difculty arises from the number of constraints that grow exponentially with the number of variables in the system. We exploit the representation of continuous-time Bayesian networks (CTBNs) to describe the Markov process. We show that by exploiting the structure of the CTBN, we can reduce the… CONTINUE READING