Introduction to the numerical solution of Markov Chains

  title={Introduction to the numerical solution of Markov Chains},
  author={William J. Stewart},
  • W. Stewart
  • Published 14 November 1994
  • Computer Science, Mathematics
* Markov Chains * Direct Methods * Iterative Methods * Projection Methods * Block Hessenberg Matrices * Decompositional Methods * LI-Cyclic Markov Chains * Transient Solutions * Stochastic Automata Networks * Software 

Matrix Analysis for Continuous-Time Markov Chains

Abstract Continuous-time Markov chains have transition matrices that vary continuously in time. Classical theory of nonnegative matrices, M-matrices and matrix exponentials is used in the literature

Error Bounds for a Stiff Markov Chain Approximation Technique and an Application

A classical stiff Markov chain solution technique is adapted to analysis of dependability models, and given a new interpretation. This allows the derivation of bounds for the approximation error. A

Algebraic Schwarz methods for the numerical solution of Markov chains

1 Numerical solution of equilibrium equations : direct methods

  • Mathematics
  • 2006
In this chapter we focus on the numerical solution of the equilibrium equations of a Markov chain with finitely many states. Consider an irreducible Markov chain with finite state space {0, 1, . . .

A dynamic programming approach for finite Markov processes and algorithms for the calculation of the limit matrix in Markov chains

New calculation procedures for finding the probabilities of state transitions of the system in discrete Markov processes based on dynamic programming are developed and polynomial time algorithms for

Efficient Computation of Equilibrium/Transient Probability Distribution of Arbitrary Finite State Space Continuous Time Markov Chains

In this research paper, efficient algorithms for computation of equilibrium as well as transient probability distribution of arbitrary finite state space Continuous / Discrete Time Markov Chains are

Algebraic Multilevel Methods for Markov Chains

A new algebraic multilevel algorithm for computing the second eigenvector of a column-stochastic matrix is presented, based on a deflation approach in a multileVEL aggregation framework, and is shown to yield good convergence properties for typical example problems.

Block Two-stage Methods for Singular Systems and Markov Chains

Research Report 95-121, Department of Mathematics, Temple University, December 1995. This paper appeared, in revised form, in Numerical Linear Algebra with Applications, vol. 3 (1996) 413-426.

A Structured Solution Approach for Markov Regenerative Processes

This paper goes one step further by even avoiding the storage of the generator matrices required by the matrix-free method, thanks to the use of a Kronecker representation.

Computational Discrete Time Markov Chain with Correlated Transition Probabilities

This study presents a computational procedure for analyzing statistics of steady state probabilities in a discrete time Markov chain with correlations among their transition probabilities. The