Finding Steady States of Communicating Markov Processes Combining Aggregation/Disaggregation with Tensor Techniques

  title={Finding Steady States of Communicating Markov Processes Combining Aggregation/Disaggregation with Tensor Techniques},
  author={Francisco Macedo},
Stochastic models for interacting processes feature a dimensionality that grows exponentially with the number of processes. This state space explosion severely impairs the use of standard methods for the numerical analysis of such Markov chains. In this work, we develop algorithms for the approximation of steady states of structured Markov chains that consider tensor train decompositions, combined with well-established techniques for this problem – aggregation/disaggregation techniques… 
1 Citations

Low-rank tensor methods for large Markov chains and forward feature selection methods

This thesis presents and compares several approaches for the determination of the steady-state of large-scale Markov chains with an underlying lowrank tensor structure, and develops a theoretical framework that allows evaluating the methods based on their theoretical properties.



Low-Rank Tensor Methods for Communicating Markov Processes

This work discusses the approximation of solutions by matrix product states or, equivalently, by tensor train decompositions, and proposes two classes of algorithms based on this low-rank decomposition that can, in principle, attain arbitrarily high accuracy.

Convergence of multi-level iterative aggregation-disaggregation methods

Multigrid Methods for Tensor Structured Markov Chains with Low Rank Approximation

This work presents an approach to adapt the algebraic multigrid framework to the Tensor frame, not only using the tensor structure in matrix-vector multiplications, but also tensor structured coarse-grid operators and tensor representations of the solution vector.

Numerical Methods in Markov Chain Modeling

This paper describes and compares several methods for computing stationary probability distributions of Markov chains based on combinations of Krylov subspace techniques, single vector power iteration/relaxation procedures and acceleration techniques.

A multi-level solution algorithm for steady-state Markov chains

A new iterative algorithm, the multi-level algorithm, for the numerical solution of steady state Markov chains is presented. The method utilizes a set of recursively coarsened representations of the

Analyzing Markov Chains using Kronecker Products: Theory and Applications

Kronecker products are used to define the underlying Markov chain (MC) in various modeling formalisms, including compositional Markovian models, hierarchical Markovian models, and stochastic process

Multigrid methods combined with low-rank approximation for tensor structured Markov chains

This work proposes a novel tensor-based algorithm that combines a tensorized multigrid method with AMEn, an optimization-based low-rank tensor solver, for addressing coarse grid problems and overcomes the limitations incurred when using each of the two methods individually.

On the Convergence of a Class of Multilevel Methods for Large Sparse Markov Chains

The theory behind the steady state analysis of large sparse Markov chains with a recently proposed class of multilevel methods using concepts from algebraic multigrid and iterative aggregation-disaggregation is investigated.

Product Form Approximations for Communicating Markov Processes

  • P. Buchholz
  • Computer Science, Mathematics
    2008 Fifth International Conference on Quantitative Evaluation of Systems
  • 2008
The paper presents the general theory of product form approximations for communicating Markov processes and it introduces first algorithms to compute product form solutions.


This document both provides an illustration of the big variety of applications associated with Stochastic Automata Network as it also allows testing and tuning algorithms for the solution of relevant associated problems as, for instance, the determination of their steady-state.