Steven I. Marcus

Learn More
This work is a survey of the average cost control problem for discrete-time Markov processes. The authors have attempted to put together a comprehensive account of the considerable research on this problem over the past three decades. The exposition ranges from finite to Borel state and action spaces and includes a variety of methodologies to find and(More)
Supremal controllable and normal sublanguages have been shown to play an important role in supervisor synthesis. In this paper, we discuss the computation of supremal controllable and normal sublanguages. We derive formulas for both supremal controllable sublanguages and supremal normal sublanguages when the languages involved are closed. As a result, those(More)
This paper studies the supervisor synthesis problem of Discrete Event Dynamical Systems (DEDS’s) through the use of synchronous composition of the plant and the supervisor and discusses some of the simplifications achieved by using the synchronous composition to model supervisory control. An alternate definition of controllability is presented. This(More)
We introduce a new randomized method called Model Reference Adaptive Search (MRAS) for solving global optimization problems. The method works with a parameterized probabilistic model on the solution space and generates at each iteration a group of candidate solutions. These candidate solutions are then used to update the parameters associated with the(More)
This paper proposes a simple analytical model called M time-scale Markov Decision Process (MMDP) for hierarchically structured sequential decision making processes, where decisions in each level in the M -level hierarchy are made in M different time-scales. In this model, the state space and the control space of each level in the hierarchy are(More)
We consider partially observable Markov decision processes with finite or countably infinite (core) state and observation spaces and finite action set. Following a standard approach, an equivalent completely observed problem is formulated, with the same finite action set but with an uncountable state space, namely the space of probability distributions on(More)