# On Skolem-hardness and saturation points in Markov decision processes

@article{Piribauer2020OnSA,
title={On Skolem-hardness and saturation points in Markov decision processes},
author={Jakob Piribauer and Christel Baier},
journal={ArXiv},
year={2020},
volume={abs/2004.11441}
}
• Published 1 April 2020
• Mathematics, Computer Science
• ArXiv
The Skolem problem and the related Positivity problem for linear recurrence sequences are outstanding number-theoretic problems whose decidability has been open for many decades. In this paper, the inherent mathematical difficulty of a series of optimization problems on Markov decision processes (MDPs) is shown by a reduction from the Positivity problem to the associated decision problems which establishes that the problems are also at least as hard as the Skolem problem as an immediate…
9 Citations

## Figures and Tables from this paper

### On the Skolem Problem and the Skolem Conjecture

• Mathematics
LICS
• 2022
It is a longstanding open problem whether there is an algorithm to decide the Skolem Problem for linear recurrence sequences (LRS) over the integers, namely whether a given such sequence has a zero

### Skolem Meets Schanuel

• Mathematics
MFCS
• 2022
The celebrated Skolem-Mahler-Lech Theorem states that the set of zeros of a linear recurrence sequence is the union of a finite set and finitely many arithmetic progressions. The corresponding

### The variance-penalized stochastic shortest path problem

• Computer Science
ICALP
• 2022
It is shown that the optimal VPE in MDPs with non-negative weights as well as an optimal deterministic finite-memory scheduler can be computed in exponential space and the threshold problem whether the maximal VPE exceeds a given rational is shown to be EXPTIME-hard and to lie in NEXPTIME.

### A Universal Skolem Set of Positive Lower Density

• Mathematics, Computer Science
MFCS
• 2022
In this work, a Skolem set of positive lower density is constructed for the class of simple LRS, a set ofpositive integers relative to which theSkolem Problem is decidable.

### On Decidability of Time-bounded Reachability in CTMDPs

• Mathematics
ICALP
• 2020
It is shown that any unconditional decidability result would imply unconditionalDecidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval.

### Universal Skolem Sets

• Mathematics, Computer Science
2021 36th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
• 2021
The notion of a Universal Skolem Set is introduced: an infinite subset of the positive integers such that there is an effective procedure that inputs a linear recurrence sequence u and decides whether u(n) = 0 for some $n \in \mathcal{S}$.

### The Orbit Problem for Parametric Linear Dynamical Systems

• Mathematics
CONCUR
• 2021
A parametric version of the Kannan-Lipton Orbit Problem for linear dynamical systems is studied, showing decidability in the case of one parameter and Skolem-hardness with two or more parameters, and a reduction from the well-knownSkolem problem for linear recurrence sequences is exhibited.

### Erratum to "Frequency Linear-time Temporal Logic"

• Mathematics
ArXiv
• 2020
This work corrects the proof of a theorem stating that satisfiability of frequency linear-time temporal logic is undecidable and states that it is correct.

## References

SHOWING 1-10 OF 50 REFERENCES

### Long-run Satisfaction of Path Properties

• Computer Science
2019 34th Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
• 2019
An exponential-time algorithm is given to compute the optimal long-run frequency of path properties for paths in Kripke structures, and the PSPACE-completeness of the threshold problem is proved.

### Reachability problems for Markov chains

• Mathematics, Computer Science
Inf. Process. Lett.
• 2015

### Stochastic Shortest Paths and Weight-Bounded Properties in Markov Decision Processes

• Mathematics
LICS
• 2018
It is shown that the problem to decide whether a disjunction of weight-bounded reachability conditions holds almost surely under some scheduler belongs to NP ∩ coNP, is solvable in pseudo-polynomial time and is at least as hard as solving two-player mean-payoff games.

### Computing Minimum and Maximum Reachability Times in Probabilistic Systems

The methods are based on algorithms to transform the original problems into problems that satisfy the required assumptions and lead to the efficient solution of two basic problems in the analysis of the reliability and performance of partially-specified systems.

### Partial and Conditional Expectations in Markov Decision Processes with Integer Weights

• Computer Science
FoSSaCS
• 2019
Two variants of the stochastic shortest path problem in Markov decision processes (MDPs) with integer weights are addressed, which constitute useful approaches to the analysis of systems without guarantees on the occurrence of an event of interest.

### On Frequency LTL in Probabilistic Systems

• Mathematics, Computer Science
CONCUR
• 2015
It is shown that for Markov chains, the problem of computing the probability with which a given fLTL formula holds has the same complexity as the analogous problem for LTL, but when restricting the frequency bound p to be 1 and negations not to be outside any G^p operator, this can be performed with the same time complexity.

### On the Skolem Problem for Continuous Linear Dynamical Systems

• Mathematics
ICALP
• 2016
This paper shows decidability of the bounded problem subject to Schanuel's Conjecture, a unifying conjecture in transcendental number theory, and analyses the unbounded problem in terms of the frequencies of the differential equation, that is, the imaginary parts of the characteristic roots.

### On Decidability of Time-bounded Reachability in CTMDPs

• Mathematics
ICALP
• 2020
It is shown that any unconditional decidability result would imply unconditionalDecidability of the bounded continuous Skolem problem, or equivalently, the problem of checking if an exponential polynomial has a non-tangential zero in a bounded interval.

### Computing Quantiles in Markov Reward Models

• Computer Science, Mathematics
FoSSaCS
• 2013
The complexity of quantile queries for until properties in discrete-time finite-state Markov decision processes with nonnegative rewards on states is studied and it is shown that qualitative quantiles queries can be evaluated in polynomial time and an exponential algorithm for the evaluation of quantitative quantile query is presented.

### One-counter Markov decision processes

• Mathematics, Computer Science
SODA '10
• 2010
The computational complexity of some central analysis problems for One-Counter Markov Decision Processes (OC-MDPs), a class of finitely-presented, countable-state MDPs, is studied and several quantitative and almost-sure limit problems can be answered in polynomial time.