Eugene A. Feinberg

Learn More
This paper deals with constrained average reward Semi-Markov Decision Processes (SMDPs) with finite state and action sets. We consider two average reward criteria. The first criterion is time-average rewards, which equal the lower limits of the expected average rewards per unit time, as the horizon tends to infinity. The second criterion is ratio-average(More)
We study an optimal admission of arriving customers to a Markovian finitecapacity queue, e.g. M/M/c/N queue, with several customer types. The system managers are paid for serving customers and penalized for rejecting them. The rewards and penalties depend on customer types. The goal is to maximize the average rewards per unit time subject to the constraint(More)
This paper deals with constrained optimization of Markov Decision Processes with a countable state space, compact action sets, continuous transition probabilities, and upper semi-continuous reward functions. The objective is to maximize the expected total discounted reward for one reward function, under several inequality constraints on similar criteria(More)
This paper deals with constrained optimization of Markov Decision Processes. Both objective function and constraints are sums of standard discounted rewards, but each with a diierent discount factor. Such models arise, e.g. in production and in applications involving multiple time scales. We prove that if a feasible policy exists, then there exists an(More)
This paper introduces and develops a new approach to the theory of continuous time jump Markov decision processes (CTJMDP). This approach reduces discounted CTJMDPs to discounted semi-Markov decision processes (SMDPs) and eventually to discrete-time Markov decision processes (MDPs). The reduction is based on the equivalence of strategies that change actions(More)
We consider a discrete time Markov Decision Process with innnite horizon. The criterion to be maximized is the sum of a number of standard discounted rewards, each with a diierent discount factor. Situations in which such criteria arise include modeling investments, production, modeling projects of diierent durations and systems with multiple criteria, and(More)
The paper studies optimization of average-reward continuous-time finite state and action Markov Decision Processes with multiple criteria and constraints. Under the standard unichain assumption, we prove the existence of optimal K-switching strategies for feasible problems with K constraints. For switching randomized strategies, the decisions depend on the(More)
The theoretical basis and quantitative evaluation of a new approach for modeling biofilm growth are presented here. Soluble components (e.g., substrates) are represented in a continuous field, whereas discrete mapping is used for solid components (e.g., biomass). The spatial distribution of substrate is calculated by applying relaxation methods to the(More)
We consider a discrete time Markov Decision Process, where the objectives are linear combinations of standard discounted rewards, each with a diierent discount factor. We describe several applications that motivate the recent interest in these criteria. For the special case where a standard discounted cost is to be minimized, subject to a constraint on(More)