Absorption in Time-Varying Markov Chains: Graph-Based Conditions

  title={Absorption in Time-Varying Markov Chains: Graph-Based Conditions},
  author={Yasin Yazıcıoğlu},
  journal={IEEE Control Systems Letters},
We investigate absorption, i.e., almost sure convergence to an absorbing state, in time-varying (non-homogeneous) discrete-time Markov chains with finite state space. We consider systems that can switch among a finite set of transition matrices, which we call the modes. Our analysis is focused on two properties: 1) almost sure convergence to an absorbing state under any switching, and 2) almost sure convergence to a desired set of absorbing states via a proper switching policy. We derive… 

Probabilistically Guaranteed Satisfaction of Temporal Logic Constraints During Reinforcement Learning

An automata-theoretic approach is proposed to ensure probabilistic satisfaction of the constraint in each episode, which is different from penalizing violations to achieve constraint satisfaction after a sufficiently large number of episodes.

Temporal-Logic-Constrained Hybrid Reinforcement Learning to Perform Optimal Aerial Monitoring with Delivery Drones

A modified Dyna-Q algorithm is proposed to address the shortage of online samples using time window temporal logic specifications to define the pickup and delivery tasks while utilizing reinforcement learning (RL) to maximize the expected sum of rewards.

BPM process implementation in a SMB Service Provider: Managing transitions risks in BPM Life Cycle with Markov Trajectories model and Gamification Techniques

This paper presents a methodological approach combining techniques from operational research and information and decision theory aligned with a gamification strategy that, when applied to the management of changes between the cycle of life BPM stages, avoids possible instabilities in corporate results.



Quasi-stationary distributions for reducible absorbing Markov chains in discrete time

We consider discrete-time Markov chains with one coffin state and a finite set $S$ of transient states, and are interested in the limiting behaviour of such a chain as time $n \to \infty,$

Conditions for weak ergodicity of inhomogeneous Markov chains

Stability of switched systems with average dwell-time

  • J. HespanhaA. Morse
  • Mathematics
    Proceedings of the 38th IEEE Conference on Decision and Control (Cat. No.99CH36304)
  • 1999
It is shown that switching among stable linear systems results in a stable system provided that switching is "slow-on-the-average". In particular, it is proved that exponential stability is achieved

Merging for time inhomogeneous finite Markov chains, Part I: singular values and stability

We develop singular value techniques in the context of time inhomogeneous finite Markov chains with the goal of obtaining quantitative results concerning the asymptotic behavior of such chains. We

Stability theory for hybrid dynamical systems

  • H. YeA. MichelL. Hou
  • Mathematics
    Proceedings of 1995 34th IEEE Conference on Decision and Control
  • 1995
Hybrid systems which are capable of exhibiting simultaneously several kinds of dynamic behavior in different parts of a system are of great current interest. In the present paper we first formulate a

Robustness of stochastic stability in game theoretic learning

In the first case, a continuity result is derived that bounds the effects of small uncertainties in the game parameters and in the second case, it is shown that game play tracks drifting stochastically stable states under sufficiently slow time variations.

Stability and Stabilizability of Switched Linear Systems: A Survey of Recent Results

This paper focuses on the stability analysis for switched linear systems under arbitrary switching, and highlights necessary and sufficient conditions for asymptotic stability.

An Introduction to Stochastic Epidemic Models

A brief introduction to the formulation of various types of stochastic epidemic models is presented based on the well-known deterministic SIS and SIR epidemic models. Three different types of

Nested Partitions Method for Global Optimization

The Nested Partitions (NP) method, a new randomized method for solving global optimization problems that systematically partitions the feasible region and concentrates the search in regions that are the most promising, is proposed.

Hybrid dynamical systems

Robust stability and control for systems that combine continuous-time and discrete-time dynamics. This article is a tutorial on modeling the dynamics of hybrid systems, on the elements of stability