This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and… (More)
Stochastic hybrid systems arise in numerous applications of systems with multiple models; e.g., air traffc management, flexible manufacturing systems, fault tolerant control systems etc. In a typical hybrid system, the state space is hybrid in the sense that some components take values in a Euclidean space, while some other components are discrete. In this… (More)
A b s t r a c t. We study the ergodic control problem of switching diffusions representing a typical hybrid system that arises in numerous applications such as fault-tolerant control systems, flexible manufacturing systems, etc. Under fairly general conditions, we establish the existence of a stable, nonrandomized Markov policy which almost surely minimizes… (More)
We study the controllability properties of the class of stochastic differential systems characterized by a linear controlled diffusion perturbed by a smooth, bounded, uniformly Lipschitz nonlinearity. We obtain conditions that guarantee the weak and strong controllability of the system. Also, given any open set in the state space we construct a control,… (More)
ABSTACT The existence of optimal stable Markov relaxed controls for the ergodic control of multidimensional diffusions is established by direct probabilistic methods based on a characterization of a.s. limit sets of empirical measures. The optimality of the above is established in the strong (i.e., almost sure) sense among all admissible controls under very… (More)
Although mouse studies have demonstrated the presence of an effector memory population in nonlymphoid tissues, the phenotype of human CD8(+) T cells present in such compartments has not been characterized. Because of the relatively large number of CD8(+) T cells present in breast milk, we were able to characterize the phenotype of this cell population in… (More)
BACKGROUND In patients with chronic renal failure (CRF), carotid artery intima media thickness (CAIMT) is increased when the patients are on hemodialysis. Vascular events caused by atherosclerosis are the major cause of death in patients undergoing hemodialysis. AIMS This study was done to find out the relationship between carotid artery intima media… (More)
We have studied two person stochastic diierential games with multiple modes. For the zero-sum game we have established the existence of optimal strategies for both players. For the nonzero sum case we have proved the existence of a Nash equilibrium. INTRODUCTION We study a two-person stochastic diierential game with multiple modes.