Survey of linear programming for standard and nonstandard Markovian control problems. Part II: Applications

@article{Kallenberg1994SurveyOL,
  title={Survey of linear programming for standard and nonstandard Markovian control problems. Part II: Applications},
  author={Lodewijk C. M. Kallenberg},
  journal={Zeitschrift f{\"u}r Operations Research},
  year={1994},
  volume={40},
  pages={127-143}
}
  • L. Kallenberg
  • Published 1 June 1994
  • Computer Science, Mathematics
  • Zeitschrift für Operations Research
This paper deals with some applications of Markov decision models for which the linear programming method is efficient. These models are replacement models (with the optimal stopping problem as special case), separable models (including the inventory model as special case) and the multi-armed bandit model. In the companion paper “Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory”, general linear programming methods are discussed. These linear… 
View on Springer
9 Citations
Highly Influential Citations
1
Background Citations
6
Methods Citations
1

9 Citations

Linear Programming Approximations for Markov Control Processes in Metric Spaces

A general framework to analyze the convergence of linear-programming approximations for Markov control processes in metric spaces is developed, under which the control problem’s optimal value can be approximated by a sequence of finite-dimensional linear programs.

Polynomial approximation method for stochastic programming.

This thesis applies SAA, polynomial approximation method and then steepest descent method in combination to solve the large-scale problems effectively and efficiently.

Finding the K best policies in a finite-horizon Markov decision process

Derman’s book as inspiration: some results on LP for MDPs

In 1976 I was looking for a suitable subject for my PhD thesis. My thesis advisor Arie Hordijk and I found a lot of inspiration in Derman’s book (Finite state Markovian decision processes, Academic…

Derman’s book as inspiration: some results on LP for MDPs

This article will describe some results in this area on the following topics: (1) MDPs with the average reward criterion; (2) additional constraints; (3) applications.

Optimal Policies for Multi-server Non-preemptive Priority Queues

  • E. Peköz
  • Business
    Queueing Syst. Theory Appl.
  • 2002
A linear programming approach is used to find and evaluate the performance of an asymptotically optimal policy in the setting of exponential service and inter-arrival times in a multi-server non-preemptive queue.

Risk-Aware Scheduling of Dual Criticality Job Systems Using Demand Distributions

A probabilistic framework for MC scheduling is developed, where feasibility is defined in terms of (chance) constraints on the probabilities that Lo and Hi jobs meet their deadlines, and those paths are dependent upon the set of execution scenarios and the given demand distributions.

High assurance real-time systems : scheduling, budgeting, and safety

This dissertation incorporates software failure rates into the scheduling problem, and synthesizes feasible scheduling policies under which the input task set will satisfy certain failure rate requirements, and model the task scheduling problem as a path-Constrained Markov Decision Process (CMDP).

Risk-Averse Control of Undiscounted Transient Markov Models

This work uses Markov risk measures to formulate a risk-averse version of the undiscounted total cost problem for a transient controlled Markov process and derives risk- averse dynamic programming equations satisfied by the optimal policy.

References

SHOWING 1-10 OF 18 REFERENCES

Survey of linear programming for standard and nonstandard Markovian control problems. Part I: Theory

This paper gives an overview of linear programming methods for solving standard and nonstandard Markovian control problems, and a particular class of stochastic games.

Linear Programming Solutions for Separable Markovian Decision Problems

This paper is concerned with the linear programming solutions to sequential decision (or control) problems in which the stochastic element is Markovian and in which the objective is to minimize the…

Linear programming and finite Markovian control problems

This text is a revised version of the author's thesis for the University of Leiden and is mainly concerned with the theory of finite Markov decision problems. Such problems are those where a decision…

Myopic Solutions of Markov Decision Processes and Stochastic Games

Sufficient conditions are presented for a Markov decision process to have a myopic optimum and for a stochastic game to possess a myopic equilibrium point. An optimum or an equilibrium point is said…

Separable Markovian decision problems

SummarySeparable Markovian decision problems have the property that for certain pairs (i, a) of a statei and an actiona: (i) the immediate reward is the sum of terms due to the current state and…

Extensions of the multiarmed bandit problem: The discounted case

A reformulation of the bandit problem yields the tax problem, which includes Klimov's waiting time problem, and an index rule is derived for the case where new machines arrive randomly.

The Multi-Armed Bandit Problem: Decomposition and Computation

It is shown that an approximate largest-index rule yields an approximately optimal policy for the N-project problem, and more efficient methods of computing the indices on-line and/or for sparse transition matrices in large state spaces than have been suggested heretofore.

Bandit processes and dynamic allocation indices

The paper aims to give a unified account of the central concepts in recent work on bandit processes and dynamic allocation indices; to show how these reduce some previously intractable problems to…

Stochastic games with state independent transitions and separable rewards

For the class of stochastic games with separable reward functions and state independent transitions it is proved that the ordered field property holds. A solution of these stochastic games can easily…

A Note on M. N. Katehakis' and Y.-R. Chen's Computation of the Gittins Index

It is shown that instead of the Kv linear programs for project v also one parametric linear program with the same dimensions can be solved.