# The computation of the mean first passage times for Markov chains

@article{Hunter2018TheCO, title={The computation of the mean first passage times for Markov chains}, author={Jeffrey J. Hunter}, journal={Linear Algebra and its Applications}, year={2018}, volume={549}, pages={100-122} }

Abstract A survey of a variety of computational procedures for finding the mean first passage times in Markov chains is presented. The author recently developed a new accurate computational technique, an Extended GTH Procedure, Hunter (2016) [17] similar to that developed by Kohlas (1986) [20] . In addition, the author recently developed a variety of new perturbation techniques for finding key properties of Markov chains including finding the mean first passage times, Hunter (2016) [18] . These… Expand

#### 16 Citations

Linear least square method for the computation of the mean first passage times of ergodic markov chains

- Mathematics
- 2018

An efficient and accurate iterative scheme for the computation of the mean first passage times (MFPTs) of ergodic Markov chains has been presented. Firstly, the computation problem of MFPTs is… Expand

Numerical analysis of first-passage processes in finite Markov chains exhibiting metastability.

- Medicine
- Physical review. E
- 2021

All properties required to analyze the first-passage path ensemble (FPPE) at both a microscopic and macroscopic level of detail can be computed using state-reduction methods, and the reactive visitation probability provides a rigorous metric to quantify the dynamical importance of a node for the productive transition between two endpoint states. Expand

Nearly reducible finite Markov chains: Theory and algorithms.

- Medicine
- The Journal of chemical physics
- 2021

All of the information required to determine the kinetically relevant transition mechanisms, and to identify the states that have a dominant effect on the global dynamics, can be computed reliably even for computationally challenging models. Expand

On State Occupancies, First Passage Times and Duration in Non-Homogeneous Semi-Markov Chains

- Mathematics
- Mathematics
- 2021

Semi-Markov processes generalize the Markov chains framework by utilizing abstract sojourn time distributions. They are widely known for offering enhanced accuracy in modeling stochastic phenomena.… Expand

GTH Algorithm, Censored Markov Chains, and $RG$-Factorization

- Mathematics
- 2021

In this paper, we provide a review on the GTH algorithm, which is a numerically stable algorithm for computing stationary probabilities of a Markov chain. Mathematically the GTH algorithm is an… Expand

Performance and stability of direct methods for computing generalized inverses of the graph Laplacian

- Mathematics
- 2020

We consider the computation of generalized inverses of the graph Laplacian for both undirected and directed graphs, with a focus on the group inverse and the closely related absorption inverse. These… Expand

Aggregation of Markov flows I: theory

- Mathematics, Medicine
- Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences
- 2018

The main contribution of the paper is to present an alternative, namely a method to aggregate groups of nodes to produce a factor, which can be iterated to make hierarchical aggregation schemes. Expand

HIGH-ORDER MULTIVARIATE MARKOV CHAIN APPLIED IN DOW JONES AND IBOVESPA INDEXES

- Economics
- Pesquisa Operacional
- 2019

In this paper we analyzed the probabilities of transitions of state between Ibovespa and Dow Jones indexes using High-order Multivariate Markov Chain. While the stock market may be profitable, the… Expand

Analysis, Modeling, and Control of Dynamic Processes in Networks

- Computer Science
- 2018

This thesis proposes an algorithm that nullifies the effect of external attacks on the opinion formation process in social networks by strategically recommending a small number of new edges to the network's users by providing a pseudo-linear time heuristic drawn upon the theory of Markov chains. Expand

Reachable Space Characterization of Markov Decision Processes with Time Variability

- Computer Science
- Robotics: Science and Systems
- 2019

The time variability property of the planning stochasticity is explored and the state reachability is investigated, and an efficient iterative method is developed that offers a good trade-off between solution optimality and time complexity. Expand

#### References

SHOWING 1-10 OF 21 REFERENCES

Simple Procedures for Finding Mean First Passage Times in Markov Chains

- Mathematics, Computer Science
- Asia Pac. J. Oper. Res.
- 2007

The derivation of mean first passage times in Markov chains involves the solution of a family of linear equations using suitable generalized inverses of the Markovian kernel I - P, where P is the transition matrix of a finite irreducible Markov chain. Expand

The Computation of Key Properties of Markov Chains via Perturbations

- Mathematics
- 2016

Abstract Computational procedures for the stationary probability distribution, the group inverse of the Markovian kernel and the mean first passage times of a finite irreducible Markov chain, are… Expand

Comparison of Some Direct Methods for Computing Stationary Distributions of Markov Chains

- Mathematics
- 1984

The purpose of this paper is to report on a comparison of an implementation of a simple direct $LU$ factorization method, suggested by Funderlic and Mankin [SIAM J. Sci. Stat. Comput., 2 (1981), pp.… Expand

ON THE MOMENTS OF MARKOV RENEWAL PROCESSES

- Mathematics
- 1969

Summary Recently Kshirsagar and Gupta [5] obtained expressions for the asymptotic values of the first two moments of a Markov renewal process. The method they employed involved formal inversion of… Expand

Generalized inverses and their application to applied probability problems

- Mathematics
- 1982

The main aim of this paper is to examine the applicability of generalized inverses to a wide variety of problems in applied probability where a Markov chain is present either directly or indirectly… Expand

Accurate Computation of the Fundamental Matrix of a Markov Chain

- Mathematics, Computer Science
- SIAM J. Matrix Anal. Appl.
- 1995

This work establishes a new representation of the fundamental matrix where matrix inversion is replaced by multiplying and then adding a pair of matrices, and shows that that can be done via back and forward substitution from numbers that have already been calculated when the GTH algorithm is used to compute the steady-state probabilities. Expand

Further comparisons of direct methods for computing stationary distributions of Markov chains

- Mathematics
- 1987

An algorithm for computing the stationary distribution of an irreducible Markov chain consisting of ergodic states is described in Grassmann et al. [Oper. Res., 33 (1985), pp. 1107–1116]. In this… Expand

Numerical Solution of Linear Equations Arising in Markov Chain Models

- Mathematics, Computer Science
- INFORMS J. Comput.
- 1989

This paper examines several methods for numerically solving linear equations that arise in the study of Markov chains and concludes that state-reduction is the most accurate and the matrix solutions have the least computation time. Expand

Generalized inverses of Markovian kernels in terms of properties of the Markov chain

- Mathematics
- 2014

Abstract All one-condition generalized inverses of the Markovian kernel I − P , where P is the transition matrix of a finite irreducible Markov chain, can be uniquely specified in terms of the… Expand

What is Fundamental for Markov Chains: First Passage Times, Fundamental Matrices, and Group Generalized Inverses

- Mathematics
- 1995

In this paper we discuss algorithms for computing the fundamental matrix, the group generalized inverse, and the mean and variance of first passage times for discrete time regular Markov chains. The… Expand