Learn More
Prerequisites and Co-requisites • Exposure to basic control concepts (e.g., EEL 4657) • Stochastic processes, with an introduction to Markov chains (e.g., EEL5544) • Working knowledge of Matlab Further information to appear at the ccc website, or contact me at meyn@ece. The broad goal is to develop tools for decision-making in complex networked systems.(More)
It is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated o.d.e. This in turn implies convergence of the algorithm. Several speciic classes of algorithms are considered as applications. It is found that the results provide (i) a simpler derivation of known results for(More)
The subject of this paper is open multiclass queueing networks, which are common models of communication networks, and complex manufacturing systems such as wafer fabrication facilities. We provide suucient conditions for the existence of bounds on long-run average moments of the queue lengths at the various stations, and we bound the rate of convergence of(More)
We consider the problems of performance analysis and stability/instability determination of queueing networks and scheduling policies. We exhibit a strong duality relationship between the performance of a system, and its stability analysis via mean drift. We obtain a variety of linear programs to conduct such stability and performance analyses. A certain(More)
—We study different notions of capacity for time-slotted ALOHA systems. In these systems, multiple users synchronously send packets in a bursty manner over a common additive white Gaussian noise (AWGN) channel. The users do not coordinate their transmissions, which may collide at the receiver. For such a system, we define both single-slot capacity and(More)
In Part I we developed stability concepts for discrete chains, together with Foster-Lyapunov criteria for them to hold. Part II was devoted to developing related stability concepts for continuous-time processes. In this paper we develop criteria for these forms of stability for continuous-parameter Markovian processes on general state spaces, based on(More)
The average cost optimal control problem is addressed for Markov decision processes with unbounded cost. It is found that the policy iteration algorithm generates a sequence of policies which are c-regular (a strong stability condition), where c is the cost function under consideration. This result only requires the existence of an initial c-regular policy,(More)
This paper establishes new criteria for stability and for instability of multiclass network models under a given stationary policy. It also extends previous results on the approximation of the solution to the average cost optimality equations through an associated fluid model: It is shown that an optimized network possesses a fluid limit model which is(More)