• Corpus ID: 117075060

Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications

@inproceedings{Yin2004DiscreteTimeMC,
  title={Discrete-Time Markov Chains: Two-Time-Scale Methods and Applications},
  author={George Yin and Qing Zhang},
  year={2004}
}
Prologue and Preliminaries.- Introduction, Overview, and Examples.- Mathematical Preliminaries.- Asymptotic Properties.- Asymptotic Expansions.- Occupation Measures.- Exponential Bounds.- Interim Summary and Extensions.- Applications.- Stability of Dynamic Systems.- Filtering.- Markov Decision Processes.- LQ Controls.- Mean-Variance Controls.- Production Planning.- Stochastic Approximation. 
No Paper Link Available
168 Citations
Highly Influential Citations
7
Background Citations
49
Methods Citations
15

168 Citations

Citation Type

Citation Type

Asymptotic properties of hybrid random processes modulated by Markov chains
Numerical Solutions of Stochastic Control Problems for Regime-switching Systems
In this paper, we develop approximation methods for solving stochastic control problems. The systems under consideration involve regime switching, modulated by a continuous-time Markov chain. Using…
Asymptotic properties of Markov-modulated random sequences with fast and slow timescales
This work is concerned with asymptotic properties of Markov-modulated random processes having two timescales. The model contains a number of mixing sequences modulated by a switching process that is…
Asymptotic Expansions for Stationary Distributions of Perturbed Semi-Markov Processes
  • D. Silvestrov, S. Silvestrov
  • Mathematics
    2016 Second International Symposium on Stochastic Models in Reliability Engineering, Life Science and Operations Management (SMRLO)
  • 2016
New algorithms for computing asymptotic expansions for power moments of hitting times and stationary and quasi-stationary distributions of nonlinearly perturbed semi-Markov processes are presented.…
Asymptotic Expansions for Stationary Distributions of Nonlinearly Perturbed Semi-Markov Processes. 1
New algorithms for construction of asymptotic expansions for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces are presented. These algorithms are based…
Asymptotic Expansions for Stationary Distributions of Nonlinearly Perturbed Semi-Markov Processes. II
Asymptotic expansions with explicit upper bounds for remainders are given for stationary distributions of nonlinearly perturbed semi-Markov processes with finite phase spaces. The corresponding…
Discrete-Time Semi-Markov Random Evolutions – Average and Diffusion Approximation of Difference Equations and Additive Functionals
This article introduces discrete-time semi-Markov random evolution and studies asymptotic properties, namely, averaging and diffusion approximation by weak convergence methods. The cases considered…
Stability of Discrete-Time Regime-Switching Dynamic Systems with Delays
TLDR
Two-time-scale formulation is used for the purpose of reduction of complexity and it is demonstrated that associated with the original system, there is a limit system that is a switching diffusion process.
Markov Decision Problems
TLDR
This chapter focuses on a class of Markov decision processes (MDPs) having weak and strong interactions, as an application of the asymptotic properties of two-time-scale Markov chains.
Discrete-time approximation of Wonham filters
TLDR
This work develops approximation schemes of Wonham filters by constructing discrete-time recursive algorithms and proves the convergence of the algorithm by weak convergence method and martingale averaging techniques.
...
...