The computation of the mean first passage times for Markov chains

@article{Hunter2018TheCO,
  title={The computation of the mean first passage times for Markov chains},
  author={Jeffrey J. Hunter},
  journal={Linear Algebra and its Applications},
  year={2018},
  volume={549},
  pages={100-122}
}
  • J. Hunter
  • Published 2018
  • Mathematics
  • Linear Algebra and its Applications
Abstract A survey of a variety of computational procedures for finding the mean first passage times in Markov chains is presented. The author recently developed a new accurate computational technique, an Extended GTH Procedure, Hunter (2016) [17] similar to that developed by Kohlas (1986) [20] . In addition, the author recently developed a variety of new perturbation techniques for finding key properties of Markov chains including finding the mean first passage times, Hunter (2016) [18] . These… Expand
Linear least square method for the computation of the mean first passage times of ergodic markov chains
An efficient and accurate iterative scheme for the computation of the mean first passage times (MFPTs) of ergodic Markov chains has been presented. Firstly, the computation problem of MFPTs isExpand
Numerical analysis of first-passage processes in finite Markov chains exhibiting metastability.
TLDR
All properties required to analyze the first-passage path ensemble (FPPE) at both a microscopic and macroscopic level of detail can be computed using state-reduction methods, and the reactive visitation probability provides a rigorous metric to quantify the dynamical importance of a node for the productive transition between two endpoint states. Expand
Nearly reducible finite Markov chains: Theory and algorithms.
TLDR
All of the information required to determine the kinetically relevant transition mechanisms, and to identify the states that have a dominant effect on the global dynamics, can be computed reliably even for computationally challenging models. Expand
On State Occupancies, First Passage Times and Duration in Non-Homogeneous Semi-Markov Chains
Semi-Markov processes generalize the Markov chains framework by utilizing abstract sojourn time distributions. They are widely known for offering enhanced accuracy in modeling stochastic phenomena.Expand
GTH Algorithm, Censored Markov Chains, and $RG$-Factorization
In this paper, we provide a review on the GTH algorithm, which is a numerically stable algorithm for computing stationary probabilities of a Markov chain. Mathematically the GTH algorithm is anExpand
Performance and stability of direct methods for computing generalized inverses of the graph Laplacian
We consider the computation of generalized inverses of the graph Laplacian for both undirected and directed graphs, with a focus on the group inverse and the closely related absorption inverse. TheseExpand
Aggregation of Markov flows I: theory
  • R. MacKay, J. Robinson
  • Mathematics, Medicine
  • Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
  • 2018
TLDR
The main contribution of the paper is to present an alternative, namely a method to aggregate groups of nodes to produce a factor, which can be iterated to make hierarchical aggregation schemes. Expand
HIGH-ORDER MULTIVARIATE MARKOV CHAIN APPLIED IN DOW JONES AND IBOVESPA INDEXES
In this paper we analyzed the probabilities of transitions of state between Ibovespa and Dow Jones indexes using High-order Multivariate Markov Chain. While the stock market may be profitable, theExpand
Analysis, Modeling, and Control of Dynamic Processes in Networks
TLDR
This thesis proposes an algorithm that nullifies the effect of external attacks on the opinion formation process in social networks by strategically recommending a small number of new edges to the network's users by providing a pseudo-linear time heuristic drawn upon the theory of Markov chains. Expand
Reachable Space Characterization of Markov Decision Processes with Time Variability
TLDR
The time variability property of the planning stochasticity is explored and the state reachability is investigated, and an efficient iterative method is developed that offers a good trade-off between solution optimality and time complexity. Expand
...
1
2
...

References

SHOWING 1-10 OF 21 REFERENCES
Simple Procedures for Finding Mean First Passage Times in Markov Chains
  • J. Hunter
  • Mathematics, Computer Science
  • Asia Pac. J. Oper. Res.
  • 2007
TLDR
The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain. Expand
The Computation of Key Properties of Markov Chains via Perturbations
Abstract Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of a finite irreducible Markov chain, areExpand
Comparison of Some Direct Methods for Computing Stationary Distributions of Markov Chains
The purpose of this paper is to report on a comparison of an implementation of a simple direct $LU$ factorization method, suggested by Funderlic and Mankin [SIAM J. Sci. Stat. Comput., 2 (1981), pp.Expand
ON THE MOMENTS OF MARKOV RENEWAL PROCESSES
Summary Recently Kshirsagar and Gupta [5] obtained expressions for the asymptotic values of the first two moments of a Markov renewal process. The method they employed involved formal inversion ofExpand
Generalized inverses and their application to applied probability problems
The main aim of this paper is to examine the applicability of generalized inverses to a wide variety of problems in applied probability where a Markov chain is present either directly or indirectlyExpand
Accurate Computation of the Fundamental Matrix of a Markov Chain
  • D. Heyman
  • Mathematics, Computer Science
  • SIAM J. Matrix Anal. Appl.
  • 1995
TLDR
This work establishes a new representation of the fundamental matrix where matrix inversion is replaced by multiplying and then adding a pair of matrices, and shows that that can be done via back and forward substitution from numbers that have already been calculated when the GTH algorithm is used to compute the steady-state probabilities. Expand
Further comparisons of direct methods for computing stationary distributions of Markov chains
An algorithm for computing the stationary distribution of an irreducible Markov chain consisting of ergodic states is described in Grassmann et al. [Oper. Res., 33 (1985), pp. 1107–1116]. In thisExpand
Numerical Solution of Linear Equations Arising in Markov Chain Models
TLDR
This paper examines several methods for numerically solving linear equations that arise in the study of Markov chains and concludes that state-reduction is the most accurate and the matrix solutions have the least computation time. Expand
Generalized inverses of Markovian kernels in terms of properties of the Markov chain
Abstract All one-condition generalized inverses of the Markovian kernel I − P , where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of theExpand
What is Fundamental for Markov Chains: First Passage Times, Fundamental Matrices, and Group Generalized Inverses
In this paper we discuss algorithms for computing the fundamental matrix, the group generalized inverse, and the mean and variance of first passage times for discrete time regular Markov chains. TheExpand
...
1
2
3
...