Learn More
c Preface A representative of a major publishing house is on her way home from a conference in Singapore, excited about the possibility of a new book series. On the flight home to New York she opens her blackberry organizer, adding names of new contacts, and is disappointed to realize she may have caught the bug that was bothering her friend Alex at the(More)
It is shown here that stability of the stochastic approximation algorithm is implied by the asymptotic stability of the origin for an associated o.d.e. This in turn implies convergence of the algorithm. Several speciic classes of algorithms are considered as applications. It is found that the results provide (i) a simpler derivation of known results for(More)
The subject of this paper is open multiclass queueing networks, which are common models of communication networks, and complex manufacturing systems such as wafer fabrication facilities. We provide suucient conditions for the existence of bounds on long-run average moments of the queue lengths at the various stations, and we bound the rate of convergence of(More)
—It is known that state-dependent, multi-step Lya-punov bounds lead to greatly simplified verification theorems for stability for large classes of Markov chain models. This is one component of the " fluid model " approach to stability of stochastic networks. In this paper we extend the general theory to random-ized multi-step Lyapunov theory to obtain(More)
This paper establishes new criteria for stability and for instability of multiclass network models under a given stationary policy. It also extends previous results on the approximation of the solution to the average cost optimality equations through an associated fluid model: It is shown that an optimized network possesses a fluid limit model which is(More)
This paper considers in parallel the scheduling problem for multi-class queueing networks, and optimization of Markov decision processes. It is shown that the value iteration algorithm may perform poorly when the algorithm is not initialized properly. The most typical case where the initial value function is taken to be zero may be a particularly bad(More)
We develop the use of piecewise linear test functions for the analysis of stability o f m ulticlass queueing networks and their associated uid limit models. It is found that if an associated LP admits a positive solution, then a Lyapunov function exists. This implies that the uid limit model is stable and hence that the network model is positive Harris(More)
—We study different notions of capacity for time-slotted ALOHA systems. In these systems, multiple users synchronously send packets in a bursty manner over a common additive white Gaussian noise (AWGN) channel. The users do not coordinate their transmissions, which may collide at the receiver. For such a system, we define both single-slot capacity and(More)
We describe an exact dynamic programming update for constrained partially observable Markov decision processes (CPOMDPs). State-of-the-art exact solution of unconstrained POMDPs relies on implicit enumeration of the vectors in the piecewise linear value function, and pruning operations to obtain a minimal representation of the updated value function. In(More)
This paper concerns the structure of optimal codes for stochastic channel models. An investigation of an associated dual convex program reveals that the optimal distribution in channel coding is typically discrete. Based on this observation we obtain the following theoretical conclusions, as well as new algorithms for constructing capacity-achieving(More)