On Skolem-hardness and saturation points in Markov decision processes

@article{Piribauer2020OnSA,
  title={On Skolem-hardness and saturation points in Markov decision processes},
  author={Jakob Piribauer and Christel Baier},
  journal={ArXiv},
  year={2020},
  volume={abs/2004.11441}
}
The Skolem problem and the related Positivity problem for linear recurrence sequences are outstanding number-theoretic problems whose decidability has been open for many decades. In this paper, the inherent mathematical difficulty of a series of optimization problems on Markov decision processes (MDPs) is shown by a reduction from the Positivity problem to the associated decision problems which establishes that the problems are also at least as hard as the Skolem problem as an immediate… 

On the Skolem Problem and the Skolem Conjecture

It is a longstanding open problem whether there is an algorithm to decide the Skolem Problem for linear recurrence sequences (LRS) over the integers, namely whether a given such sequence has a zero

Skolem Meets Schanuel

The celebrated Skolem-Mahler-Lech Theorem states that the set of zeros of a linear recurrence sequence is the union of a finite set and finitely many arithmetic progressions. The corresponding

The variance-penalized stochastic shortest path problem

It is shown that the optimal VPE in MDPs with non-negative weights as well as an optimal deterministic finite-memory scheduler can be computed in exponential space and the threshold problem whether the maximal VPE exceeds a given rational is shown to be EXPTIME-hard and to lie in NEXPTIME.

A Universal Skolem Set of Positive Lower Density

In this work, a Skolem set of positive lower density is constructed for the class of simple LRS, a set ofpositive integers relative to which theSkolem Problem is decidable.

On Decidability of Time-bounded Reachability in CTMDPs

It is shown that any unconditional decidability result would imply unconditionalDecidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval.

Universal Skolem Sets

The notion of a Universal Skolem Set is introduced: an infinite subset of the positive integers such that there is an effective procedure that inputs a linear recurrence sequence u and decides whether u(n) = 0 for some $n \in \mathcal{S}$.

The Orbit Problem for Parametric Linear Dynamical Systems

A parametric version of the Kannan-Lipton Orbit Problem for linear dynamical systems is studied, showing decidability in the case of one parameter and Skolem-hardness with two or more parameters, and a reduction from the well-knownSkolem problem for linear recurrence sequences is exhibited.

Erratum to "Frequency Linear-time Temporal Logic"

This work corrects the proof of a theorem stating that satisfiability of frequency linear-time temporal logic is undecidable and states that it is correct.

References

SHOWING 1-10 OF 50 REFERENCES

Long-run Satisfaction of Path Properties

An exponential-time algorithm is given to compute the optimal long-run frequency of path properties for paths in Kripke structures, and the PSPACE-completeness of the threshold problem is proved.

Reachability problems for Markov chains

Stochastic Shortest Paths and Weight-Bounded Properties in Markov Decision Processes

It is shown that the problem to decide whether a disjunction of weight-bounded reachability conditions holds almost surely under some scheduler belongs to NP ∩ coNP, is solvable in pseudo-polynomial time and is at least as hard as solving two-player mean-payoff games.

Computing Minimum and Maximum Reachability Times in Probabilistic Systems

The methods are based on algorithms to transform the original problems into problems that satisfy the required assumptions and lead to the efficient solution of two basic problems in the analysis of the reliability and performance of partially-specified systems.

Partial and Conditional Expectations in Markov Decision Processes with Integer Weights

Two variants of the stochastic shortest path problem in Markov decision processes (MDPs) with integer weights are addressed, which constitute useful approaches to the analysis of systems without guarantees on the occurrence of an event of interest.

On Frequency LTL in Probabilistic Systems

It is shown that for Markov chains, the problem of computing the probability with which a given fLTL formula holds has the same complexity as the analogous problem for LTL, but when restricting the frequency bound p to be 1 and negations not to be outside any G^p operator, this can be performed with the same time complexity.

On the Skolem Problem for Continuous Linear Dynamical Systems

This paper shows decidability of the bounded problem subject to Schanuel's Conjecture, a unifying conjecture in transcendental number theory, and analyses the unbounded problem in terms of the frequencies of the differential equation, that is, the imaginary parts of the characteristic roots.

On Decidability of Time-bounded Reachability in CTMDPs

It is shown that any unconditional decidability result would imply unconditionalDecidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval.

Computing Quantiles in Markov Reward Models

The complexity of quantile queries for until properties in discrete-time finite-state Markov decision processes with nonnegative rewards on states is studied and it is shown that qualitative quantiles queries can be evaluated in polynomial time and an exponential algorithm for the evaluation of quantitative quantile query is presented.

One-counter Markov decision processes

The computational complexity of some central analysis problems for One-Counter Markov Decision Processes (OC-MDPs), a class of finitely-presented, countable-state MDPs, is studied and several quantitative and almost-sure limit problems can be answered in polynomial time.