Learn More
We study an optimal admission of arriving customers to a Markovian finite-capacity queue, e.g. M/M/c/N queue, with several customer types. The system managers are paid for serving customers and penalized for rejecting them. The rewards and penalties depend on customer types. The goal is to maximize the average rewards per unit time subject to the constraint(More)
This paper introduces and develops a new approach to the theory of continuous time jump Markov decision processes (CTJMDP). This approach reduces discounted CTJMDPs to discounted semi-Markov decision processes (SMDPs) and eventually to discrete-time Markov decision processes (MDPs). The reduction is based on the equivalence of strategies that change actions(More)
The paper studies optimization of average-reward continuous-time finite state and action Markov Decision Processes with multiple criteria and constraints. Under the standard unichain assumption, we prove the existence of optimal K-switching strategies for feasible problems with K constraints. For switching ran-domized strategies, the decisions depend on the(More)
This paper studies a discrete-time total-reward Markov decision process (MDP) with a given initial state distribution. A (randomized) stationary policy can be split on a given set of states if the occupancy measure of this policy can be expressed as a convex combination of the occupancy measures of stationary policies, each selecting deterministic actions(More)
This paper deals with constrained optimization of Markov Decision Processes. Both objective function and constraints are sums of standard discounted rewards, but each with a diierent discount factor. Such models arise, e.g. in production and in applications involving multiple time scales. We prove that if a feasible policy exists, then there exists an(More)
We consider a discrete time Markov Decision Process, where the objectives are linear combinations of standard discounted rewards, each with a diierent discount factor. We describe several applications that motivate the recent interest in these criteria. For the special case where a standard discounted cost is to be minimized, subject to a constraint on(More)
The paper deals with the quickest detection of a change of the drift of the Brownian motion. We show that the generalized Bayesian formulation of the quickest detection problem can be reduced to the optimal stopping problem for a diffusion Markov process. For this problem the optimal procedure is described and its characteristics are found. We show also(More)