# Introduction to the numerical solution of Markov Chains

@inproceedings{Stewart1994IntroductionTT, title={Introduction to the numerical solution of Markov Chains}, author={William J. Stewart}, year={1994} }

* Markov Chains * Direct Methods * Iterative Methods * Projection Methods * Block Hessenberg Matrices * Decompositional Methods * LI-Cyclic Markov Chains * Transient Solutions * Stochastic Automata Networks * Software

## 1,926 Citations

### Matrix Analysis for Continuous-Time Markov Chains

- MathematicsSpecial Matrices
- 2021

Abstract Continuous-time Markov chains have transition matrices that vary continuously in time. Classical theory of nonnegative matrices, M-matrices and matrix exponentials is used in the literature…

### Error Bounds for a Stiff Markov Chain Approximation Technique and an Application

- Mathematics, Computer Science
- 1999

A classical stiff Markov chain solution technique is adapted to analysis of dependability models, and given a new interpretation. This allows the derivation of bounds for the approximation error. A…

### Algebraic Schwarz methods for the numerical solution of Markov chains

- Mathematics, Computer Science
- 2004

### 1 Numerical solution of equilibrium equations : direct methods

- Mathematics
- 2006

In this chapter we focus on the numerical solution of the equilibrium equations of a Markov chain with finitely many states. Consider an irreducible Markov chain with finite state space {0, 1, . . .…

### A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains

- Mathematics, Computer Science
- 2011

New calculation procedures for finding the probabilities of state transitions of the system in discrete Markov processes based on dynamic programming are developed and polynomial time algorithms for…

### Efficient Computation of Equilibrium/Transient Probability Distribution of Arbitrary Finite State Space Continuous Time Markov Chains

- Mathematics
- 2020

In this research paper, efficient algorithms for computation of equilibrium as well as transient probability distribution of arbitrary finite state space Continuous / Discrete Time Markov Chains are…

### Algebraic Multilevel Methods for Markov Chains

- Computer Science
- 2017

A new algebraic multilevel algorithm for computing the second eigenvector of a column-stochastic matrix is presented, based on a deflation approach in a multileVEL aggregation framework, and is shown to yield good convergence properties for typical example problems.

### Block Two-stage Methods for Singular Systems and Markov Chains

- MathematicsNumer. Linear Algebra Appl.
- 1996

Research Report 95-121, Department of Mathematics, Temple University, December 1995. This paper appeared, in revised form, in Numerical Linear Algebra with Applications, vol. 3 (1996) 413-426.

### A Structured Solution Approach for Markov Regenerative Processes

- Computer ScienceQEST
- 2014

This paper goes one step further by even avoiding the storage of the generator matrices required by the matrix-free method, thanks to the use of a Kronecker representation.

### Computational Discrete Time Markov Chain with Correlated Transition Probabilities

- Mathematics
- 2006

This study presents a computational procedure for analyzing statistics of steady state probabilities in a discrete time Markov chain with correlations among their transition probabilities. The…