Trace Refinement in Labelled Markov Decision Processes

  title={Trace Refinement in Labelled Markov Decision Processes},
  author={Nathana{\"e}l Fijalkow and S. Kiefer and M. Shirmohammadi},
Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced labelled Markov chains are trace-equivalent. We show that this problem is decidable in polynomial time if the second MDP is a Markov chain. The algorithm is based on new results on a particular notion of bisimulation between distributions over the states. However, we show that the general trace… Expand
Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the inducedExpand
Comparing Labelled Markov Decision Processes
This work provides the first polynomial-time algorithms for computing memoryless strategies to make the two labelled MDPs inequivalent if such strategies exist, and studies the computational complexity of qualitative problems about making the total variation distance and the probabilistic bisimilarity distance less than one or equal to one. Expand
The Complexity of Synchronizing Markov Decision Processes
This work considers Markov decision processes (MDPs) as generators of sequences of probability distributions over states, and establishes matching upper and lower complexity bounds of the problems. Expand
Behavioural Preorders on Stochastic Systems - Logical, Topological, and Computational Aspects
This work introduces a logical formalism for reasoning about upper and lower bounds on time, and studies the properties of this formalism, including axiomatisation and algorithms for checking when a formula is satisfied, and considers the question of when a system is faster than another system. Expand
Survey of Statistical Verification of Linear Unbounded Properties: Model Checking and Distances
This work discusses statistical model checking of Markov chains and Markov decision processes against reachability, unbounded-until, LTL and mean-payoff properties, and how the respective strategies can be represented efficiently using statistical techniques. Expand
Linear Distances between Markov Chains
A general class of distances (metrics) between Markov chains, which are based on linear behaviour are introduced, and which of the distances can be approximated by observing the systems, i.e. by black-box testing or simulation. Expand
Computation of Kullback-Leibler Divergence Between Labeled Stochastic Systems with Non-identical State Spaces
A fixed point polynomial time algorithm is presented to compute Kullback-Leibler divergence with an approximation when comparing labeled stochastic systems with non-identical state spaces. Expand
On Restricted Nonnegative Matrix Factorization
It is shown that restricted NMF is closely related to a question about the nature of minimal probabilistic automata, posed by Paz in his seminal 1971 textbook, and this connection is used to answer Paz's question negatively, thus falsifying a positive answer claimed in 1974. Expand
103 : 2 On Restricted Nonnegative Matrix Factorization
Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative n × m matrix M into a product of a nonnegative n × d matrix W and a nonnegative d × m matrix H. Restricted NMFExpand
On Rationality of Nonnegative Matrix Factorization
It is shown that state minimization of labeled Markov chains can require the introduction of irrational transition probabilities, and complements this result with an NP-complete version of NMF for which rational numbers suffice. Expand


Equivalence of Labeled Markov Chains
It is shown that equivalence can be decided in polynomial time, using a reduction to the equivalence problem for probabilistic automata, which is known to be solvable in poynomial time. Expand
Probabilistic Bisimulation: Naturally on Distributions
This paper gives a natural definition of strong bisimulation for probabilistic systems corresponding to this view that treats probability distributions as first-class citizens and solves a longstanding open problem concerning the representation of memoryless continuous time by memoryfull continuous time. Expand
Quantum Markov chains: Description of hybrid systems, decidability of equivalence, and model checking linear-time properties
A model of quantum Markov chains is studied that is a quantum analogue of Markov Chains and is obtained by replacing probabilities in transition matrices with quantum operations and is shown to be decidable in polynomial time. Expand
On Path Equivalence of Nondeterministic Finite Automata
  • W. Tzeng
  • Mathematics, Computer Science
  • Inf. Process. Lett.
  • 1996
The path equivalence problem for NFAs without λ-cycles is solvable not only in polynomial sequential time but also in O(log2(n) parallel time using a polynogeneous number of processors, where n is the total number of states in two NFAs. Expand
On the complexity of space bounded interactive proofs
  • A. Condon, R. Lipton
  • Mathematics, Computer Science
  • 30th Annual Symposium on Foundations of Computer Science
  • 1989
The proof method for the lower bound shows that the emptiness problem for one-way probabilistic finite-state machines is undecidable and some results of independent interest on the rate of convergence of time-varying Markov chains to their halting states are obtained. Expand
Approximate Verification and Enumeration Problems
This work considers the enumeration of monomials of a polynomial given as a black box, and the enumerations of discrete points which separate two polytopes in a space of dimension n, using a random walk to enumerate words which e-distinguish two nondeterministic finite automata. Expand
When Equivalence and Bisimulation Join Forces in Probabilistic Automata
This paper considers Segala's automata, and proposes a novel notion of distribution-based bisimulation by joining the existing equivalence and bisimilarities, which gives rise to a robust notion of equivalence for Rabin's Automata. Expand
APEX: An Analyzer for Open Probabilistic Programs
APEX can verify a broad range of anonymity and termination properties of randomised protocols and other open programs, sometimes with an exponential speed-up over competing state-of-the-art approaches. Expand
Language Equivalence for Probabilistic Automata
A new randomised algorithm for deciding language equivalence for probabilistic automata based on polynomial identity testing and thus returns an answer with an error probability that can be made arbitrarily small is proposed. Expand
Reachability in two-clock timed automata is PSPACE-complete
It is shown that reachability in bounded one-counter automata is PSPACE-complete, and the previous result was shown to be log-space equivalent to reachable in boundedOne-Counter automata. Expand