Learn More
We consider dynamic, two-player, zero-sum games where the minimizing" player seeks to drive an underlying nite-state dynamic system to a special terminal state along a least expected cost path. The maximizer" seeks to interfere with the minimizer's progress so as to maximize the expected total cost. We consider, for the rst time, undiscounted nite-state(More)
— The network calculus offers an elegant framework for determining worst-case bounds on delay and backlog in a network. This paper extends the network calculus to a probabilistic framework with statistical service guarantees. The notion of a statistical service curve is presented as a probabilistic bound on the service received by an individual flow or an(More)
The deterministic network calculus offers an elegant framework for determining delays and backlog in a network with deterministic service guarantees to individual traffic flows. This paper addresses the problem of extending the network calculus to a probabilistic framework with statistical service guarantees. Here, the key difficulty relates to expressing,(More)
We consider a class of undiscounted terminating Markov decision processes with a risk-averse exponential objective function and compact constraint sets. After assuming the existence of an absorbing cost-free terminal state , positive transition costs away from , and continuity of the transition probability and cost functions, we establish (i) the existence(More)
We present a computational case study of neuro-dynamic programming , a recent class of reinforcement learning methods. We cast the problem of play selection in American football as a stochastic shortest path Markov Decision Problem MDP. In particular, we consider the problem faced by a quarterback in attempting to maximize the net score of an ooensive(More)
Scalability concerns of QoS implementations have stipulated service architectures where QoS is not provisioned separately to each flow, but instead to aggregates of flows. This paper determines stochastic bounds for the service experienced by a single flow when resources are managed for aggregates of flows and when the scheduling algorithms used in the(More)
Recent research on statistical multiplexing has provided many new insights into the achievable mul-tiplexing gain in QoS networks, however, generally only in terms of the gain experienced at a single switch. Evaluating the statistical multiplexing gain in a general network remains a difficult challenge. In this paper we describe two distinct network designs(More)
| The purpose of this paper is to propose a solution methodology for a missile defense problem involving the sequential allocation of defensive resources over a series of engagements. The problem is cast as a dynamic program-ming/Markovian decision problem, which is computation-ally intractable by exact methods because of its large number of states and its(More)
We consider the problem of allocating bandwidth to competing flows in an MPLS network, subject to constraints on fairness, efficiency, and administrative complexity. The aggregate traffic between a source and a destination, called a flow, is mapped to label switched paths (LSPs) across the network. Each flow is assigned a preferred ('primary') LSP, but(More)