Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach

  title={Continuous-Time Markov Chains and Applications: A Singular Perturbation Approach},
  author={George Yin and Qing Zhang},
Prologue and Preliminaries: Introduction and overview- Mathematical preliminaries. Markovian models.- Singularly perturbed Markov chains: Asymptotic expansion: Irreducible generators. Asymptotic normality and exponential bounds. Asymptotic expansion: Weak and strong interactions. Weak and strong interactions: Asymptotic properties and ramification.- Optimizations and numerical methods: Markov decision problems. Stochastic control of dynamical systems. Numerical methods for control and… 
Singularly perturbed Markov chains. I. Asymptotic properties
  • G. Yin, Q. Zhang
  • Mathematics
    Proceedings of the 36th IEEE Conference on Decision and Control
  • 1997
This work is concerned with singularly perturbed Markov chains. Its focus is on asymptotic properties, and applications for controlled dynamic systems. An integrated approach to continuous-time
Discrete-time Singularly Perturbed Markov Chains
This chapter provides a survey on recent developments for the study of singularly perturbed Markov chains in discrete time. We illustrate the use of singular perturbation techniques to treat
Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes
  • D. Silvestrov, S. Silvestrov
  • Mathematics
    2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO)
  • 2016
New algorithms for computing asymptotic expansions for power moments of hitting times and stationary and quasi-stationary distributions of nonlinearly perturbed semi-Markov processes are presented.
Asymptotic Expansions for Moment Functionals of Perturbed Discrete Time Semi-Markov Processes
In this paper we study moment functionals of mixed power-exponential type for non-linearly perturbed semi-Markov processes in discrete time. Conditions under which the moment functionals of interest
Control of singularly perturbed Markov chains: A numerical study
Numerical studies of nearly optimal controls of systems driven by singularly perturbed Markov chains based on the ideas of hierarchical controls applicable to many large-scale systems provide insights for further applications.
Asymptotic expansions of backward equations for two-time-scale Markov chains in continuous time
This work develops asymptotic expansions for solutions of systems of backward equations of time-inhomogeneous Markov chains in continuous time. Owing to the rapid progress in technology and the
Nonlinearly Perturbed Stochastic Processes and Systems
This paper is a survey of results presented in the recent book Gyllenberg and Silvestrov [GS08].1 This book is devoted to studies of quasi-stationary phenomena for nonlinearly perturbed stochastic