#### Filter Results:

#### Publication Year

1997

2016

#### Publication Type

#### Co-author

#### Publication Venue

#### Key Phrases

Learn More

We consider dynamic, two-player, zero-sum games where the minimizing" player seeks to drive an underlying nite-state dynamic system to a special terminal state along a least expected cost path. The maximizer" seeks to interfere with the minimizer's progress so as to maximize the expected total cost. We consider, for the rst time, undiscounted nite-state… (More)

The deterministic network calculus offers an elegant framework for determining delays and backlog in a network with deterministic service guarantees to individual traffic flows. This paper addresses the problem of extending the network calculus to a probabilistic framework with statistical service guarantees. Here, the key difficulty relates to expressing,… (More)

- Stephen D Patek
- 1999

We analyze a class of partially observed stochastic shortest path problems. These are terminating Markov decision process with imperfect state information that evolve on an innnite time horizon and have a total cost criterion. For well-posedness, we make reasonable stochastic shortest path type assumptions: (1) the existence of a policy that guarantees… (More)

- Stephen D. Patek
- 2008

• Allocate defense resources to maximize the value of assets that remain after a battle. – Attacks (e.g. SCUDs) come in waves. – We must assign defenses (e.g. Patriot missiles) so as to minimize overall damage incurred. • Sequential decisions • State feedback • Probabilistic uncertainty • Additive cost • Optimality reflects a tradeoff between short-term and… (More)

— The network calculus offers an elegant framework for determining worst-case bounds on delay and backlog in a network. This paper extends the network calculus to a probabilistic framework with statistical service guarantees. The notion of a statistical service curve is presented as a probabilistic bound on the service received by an individual flow or an… (More)

Modularity plays a key role in many engineering systems, allowing for plug-and-play integration of components, enhancing flexibility and adaptability, and facilitating standardization. In the control of diabetes, i.e., the so-called "artificial pancreas," modularity allows for the step-wise introduction of (and regulatory approval for) algorithmic… (More)

We consider a class of undiscounted terminating Markov decision processes with a risk-averse exponential objective function and compact constraint sets. After assuming the existence of an absorbing cost-free terminal state , positive transition costs away from , and continuity of the transition probability and cost functions, we establish (i) the existence… (More)

We present a computational case study of neuro-dynamic programming , a recent class of reinforcement learning methods. We cast the problem of play selection in American football as a stochastic shortest path Markov Decision Problem MDP. In particular, we consider the problem faced by a quarterback in attempting to maximize the net score of an ooensive… (More)