# Absorption in Time-Varying Markov Chains: Graph-Based Conditions

@article{Yazcolu2020AbsorptionIT,
title={Absorption in Time-Varying Markov Chains: Graph-Based Conditions},
author={Yasin Yazıcıoğlu},
journal={IEEE Control Systems Letters},
year={2020},
volume={5},
pages={1127-1132}
}
We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition matrices, which we call the modes. Our analysis is focused on two properties: 1) almost sure convergence to an absorbing state under any switching, and 2) almost sure convergence to a desired set of absorbing states via a proper switching policy. We derive…
3 Citations
• Computer Science
2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)
• 2021
An automata-theoretic approach is proposed to ensure probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve constraint satisfaction after a sufficiently large number of episodes.
• Computer Science
2021 International Conference on Unmanned Aircraft Systems (ICUAS)
• 2021
A modified Dyna-Q algorithm is proposed to address the shortage of online samples using time window temporal logic specifications to define the pickup and delivery tasks while utilizing reinforcement learning (RL) to maximize the expected sum of rewards.
• Computer Science
• 2020
This paper presents a methodological approach combining techniques from operational research and information and decision theory aligned with a gamification strategy that, when applied to the management of changes between the cycle of life BPM stages, avoids possible instabilities in corporate results.

## References

SHOWING 1-10 OF 21 REFERENCES

• Mathematics
• 2008
We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \to \infty,$
• Mathematics
Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304)
• 1999
It is shown that switching among stable linear systems results in a stable system provided that switching is "slow-on-the-average". In particular, it is proved that exponential stability is achieved
• Mathematics
• 2009
We develop singular value techniques in the context of time inhomogeneous finite Markov chains with the goal of obtaining quantitative results concerning the asymptotic behavior of such chains. We
• Mathematics
Proceedings of 1995 34th IEEE Conference on Decision and Control
• 1995
Hybrid systems which are capable of exhibiting simultaneously several kinds of dynamic behavior in different parts of a system are of great current interest. In the present paper we first formulate a
• Mathematics
2013 American Control Conference
• 2013
In the first case, a continuity result is derived that bounds the effects of small uncertainties in the game parameters and in the second case, it is shown that game play tracks drifting stochastically stable states under sufficiently slow time variations.
• Mathematics
IEEE Transactions on Automatic Control
• 2009
This paper focuses on the stability analysis for switched linear systems under arbitrary switching, and highlights necessary and sufficient conditions for asymptotic stability.
A brief introduction to the formulation of various types of stochastic epidemic models is presented based on the well-known deterministic SIS and SIR epidemic models. Three different types of
• Computer Science
Oper. Res.
• 2000
The Nested Partitions (NP) method, a new randomized method for solving global optimization problems that systematically partitions the feasible region and concentrates the search in regions that are the most promising, is proposed.
• Mathematics
IEEE Control Systems
• 2009
Robust stability and control for systems that combine continuous-time and discrete-time dynamics. This article is a tutorial on modeling the dynamics of hybrid systems, on the elements of stability