# Trace Refinement in Labelled Markov Decision Processes

@inproceedings{Fijalkow2016TraceRI, title={Trace Refinement in Labelled Markov Decision Processes}, author={Nathana{\"e}l Fijalkow and S. Kiefer and M. Shirmohammadi}, booktitle={FoSSaCS}, year={2016} }

Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced labelled Markov chains are trace-equivalent. We show that this problem is decidable in polynomial time if the second MDP is a Markov chain. The algorithm is based on new results on a particular notion of bisimulation between distributions over the states. However, we show that the general trace… Expand

#### 10 Citations

L O ] 2 8 N ov 2 01 8 TRACE REFINEMENT IN LABELLED MARKOV DECISION PROCESSES

- 2018

Given two labelled Markov decision processes (MDPs), the trace-refinement problem asks whether for all strategies of the first MDP there exists a strategy of the second MDP such that the induced… Expand

Comparing Labelled Markov Decision Processes

- Computer Science
- FSTTCS
- 2020

This work provides the first polynomial-time algorithms for computing memoryless strategies to make the two labelled MDPs inequivalent if such strategies exist, and studies the computational complexity of qualitative problems about making the total variation distance and the probabilistic bisimilarity distance less than one or equal to one. Expand

The Complexity of Synchronizing Markov Decision Processes

- Mathematics, Computer Science
- J. Comput. Syst. Sci.
- 2019

This work considers Markov decision processes (MDPs) as generators of sequences of probability distributions over states, and establishes matching upper and lower complexity bounds of the problems. Expand

Behavioural Preorders on Stochastic Systems - Logical, Topological, and Computational Aspects

- Computer Science, Mathematics
- ArXiv
- 2019

This work introduces a logical formalism for reasoning about upper and lower bounds on time, and studies the properties of this formalism, including axiomatisation and algorithms for checking when a formula is satisfied, and considers the question of when a system is faster than another system. Expand

Survey of Statistical Verification of Linear Unbounded Properties: Model Checking and Distances

- Computer Science
- ISoLA
- 2016

This work discusses statistical model checking of Markov chains and Markov decision processes against reachability, unbounded-until, LTL and mean-payoff properties, and how the respective strategies can be represented efficiently using statistical techniques. Expand

Linear Distances between Markov Chains

- Mathematics, Computer Science
- CONCUR
- 2016

A general class of distances (metrics) between Markov chains, which are based on linear behaviour are introduced, and which of the distances can be approximated by observing the systems, i.e. by black-box testing or simulation. Expand

Computation of Kullback-Leibler Divergence Between Labeled Stochastic Systems with Non-identical State Spaces

- Computer Science
- AAIM
- 2018

A fixed point polynomial time algorithm is presented to compute Kullback-Leibler divergence with an approximation when comparing labeled stochastic systems with non-identical state spaces. Expand

On Restricted Nonnegative Matrix Factorization

- Mathematics, Computer Science
- ICALP
- 2016

It is shown that restricted NMF is closely related to a question about the nature of minimal probabilistic automata, posed by Paz in his seminal 1971 textbook, and this connection is used to answer Paz's question negatively, thus falsifying a positive answer claimed in 1974. Expand

103 : 2 On Restricted Nonnegative Matrix Factorization

- 2016

Nonnegative matrix factorization (NMF) is the problem of decomposing a given nonnegative n × m matrix M into a product of a nonnegative n × d matrix W and a nonnegative d × m matrix H. Restricted NMF… Expand

On Rationality of Nonnegative Matrix Factorization

- Computer Science, Mathematics
- SODA
- 2017

It is shown that state minimization of labeled Markov chains can require the introduction of irrational transition probabilities, and complements this result with an NP-complete version of NMF for which rational numbers suffice. Expand

#### References

SHOWING 1-10 OF 38 REFERENCES

Equivalence of Labeled Markov Chains

- Computer Science, Mathematics
- Int. J. Found. Comput. Sci.
- 2008

It is shown that equivalence can be decided in polynomial time, using a reduction to the equivalence problem for probabilistic automata, which is known to be solvable in poynomial time. Expand

Probabilistic Bisimulation: Naturally on Distributions

- Mathematics, Computer Science
- CONCUR
- 2014

This paper gives a natural definition of strong bisimulation for probabilistic systems corresponding to this view that treats probability distributions as first-class citizens and solves a longstanding open problem concerning the representation of memoryless continuous time by memoryfull continuous time. Expand

Quantum Markov chains: Description of hybrid systems, decidability of equivalence, and model checking linear-time properties

- Computer Science, Physics
- Inf. Comput.
- 2015

A model of quantum Markov chains is studied that is a quantum analogue of Markov Chains and is obtained by replacing probabilities in transition matrices with quantum operations and is shown to be decidable in polynomial time. Expand

On Path Equivalence of Nondeterministic Finite Automata

- Mathematics, Computer Science
- Inf. Process. Lett.
- 1996

The path equivalence problem for NFAs without λ-cycles is solvable not only in polynomial sequential time but also in O(log2(n) parallel time using a polynogeneous number of processors, where n is the total number of states in two NFAs. Expand

On the complexity of space bounded interactive proofs

- Mathematics, Computer Science
- 30th Annual Symposium on Foundations of Computer Science
- 1989

The proof method for the lower bound shows that the emptiness problem for one-way probabilistic finite-state machines is undecidable and some results of independent interest on the rate of convergence of time-varying Markov chains to their halting states are obtained. Expand

Approximate Verification and Enumeration Problems

- Mathematics, Computer Science
- ICTAC
- 2012

This work considers the enumeration of monomials of a polynomial given as a black box, and the enumerations of discrete points which separate two polytopes in a space of dimension n, using a random walk to enumerate words which e-distinguish two nondeterministic finite automata. Expand

When Equivalence and Bisimulation Join Forces in Probabilistic Automata

- Mathematics, Computer Science
- FM
- 2014

This paper considers Segala's automata, and proposes a novel notion of distribution-based bisimulation by joining the existing equivalence and bisimilarities, which gives rise to a robust notion of equivalence for Rabin's Automata. Expand

APEX: An Analyzer for Open Probabilistic Programs

- Computer Science
- CAV
- 2012

APEX can verify a broad range of anonymity and termination properties of randomised protocols and other open programs, sometimes with an exponential speed-up over competing state-of-the-art approaches. Expand

Language Equivalence for Probabilistic Automata

- Computer Science
- CAV
- 2011

A new randomised algorithm for deciding language equivalence for probabilistic automata based on polynomial identity testing and thus returns an answer with an error probability that can be made arbitrarily small is proposed. Expand

Reachability in two-clock timed automata is PSPACE-complete

- Computer Science
- Inf. Comput.
- 2013

It is shown that reachability in bounded one-counter automata is PSPACE-complete, and the previous result was shown to be log-space equivalent to reachable in boundedOne-Counter automata. Expand