Sound Value Iteration

@inproceedings{Quatmann2018SoundVI,
  title={Sound Value Iteration},
  author={Tim Quatmann and Joost-Pieter Katoen},
  booktitle={CAV},
  year={2018}
}
Computing reachability probabilities is at the heart of probabilistic model checking. All model checkers compute these probabilities in an iterative fashion using value iteration. This technique approximates a fixed point from below by determining reachability probabilities for an increasing number of steps. To avoid results that are significantly off, variants have recently been proposed that converge from both below and above. These procedures require starting values for both sides. We… 
Optimistic Value Iteration
TLDR
This paper obtains a lower bound via standard value iteration, uses the result to “guess” an upper bound, and proves the latter’s correctness, and presents this optimistic value iteration approach for computing reachability probabilities as well as expected rewards.
Symblicit exploration and elimination for probabilistic model checking
TLDR
This paper presents a new "symblicit" approach to checking Markov chains and related probabilistic models: a decision diagram that symbolically collects all reachable states and their predecessors and eliminates few explicit states from the explicit state space in a way that preserves all relevant probabilities and rewards.
Correct Probabilistic Model Checking with Floating-Point Arithmetic
TLDR
This paper shows how to implement fast and correct probabilistic model checking by exploiting the ability of current hardware to control the direction of rounding in floating-point calculations, and outlines the complications in achieving correct rounding from higherlevel programming languages.
Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes
TLDR
This paper focuses on the computation of the expected rewards of models and proposes two heuristics to improve the performance of the interval iteration method and proposes a criterion for the correctness of the approximated upper bound.
Stopping Criteria for Value and Strategy Iteration on Concurrent Stochastic Reachability Games
TLDR
This work considers concurrent stochastic games played on graphs with reachability and safety objectives, and provides the first (anytime) algorithms with stopping criteria for value iteration and strategy iteration.
A Modest Approach to Modelling and Checking Markov Automata
TLDR
Extensions to the Modest language and the mcsta model checker are presented to describe and analyse Markov automata models and it is shown that mcsta improves the performance and scalability of Markov Automata model checking compared to earlier and alternative tools.
Verification of Multiplayer Stochastic Games via Abstract Dependency Graphs
TLDR
An efficient model checking algorithm for alternating-time temporal logic (ATL) on turn-based multiplayer stochastic games with weighted transitions and provides an efficient reduction of the model checking problem to finding the minimum fixed-point assignment on an ADG over the domain of unit intervals extended with certain-zero optimization.
A Modest Approach to Markov Automata
TLDR
Extensions to MODEST, an expressive high-level language with roots in process algebra, are presented that allow large Markov automata models to be specified in a succinct, modular way and illustrate the advantages of MODEST over alternative languages.
The Probabilistic Model Checker Storm
TLDR
The main features of Storm are reported and how to effectively use them are explained and an empirical evaluation of different configurations of Storm on the QComp 2019 benchmark set is presented.
On Correctness, Precision, and Performance in Quantitative Verification - QComp 2020 Competition Report
TLDR
This paper surveys the precision guarantees—ranging from exact rational results to statistical confidence statements—offered by the nine participating tools and reports on the experimental evaluation of these trade-offs performed in QComp 2020: the second friendly competition of tools for the analysis of quantitative formal models.
...
...

References

SHOWING 1-10 OF 25 REFERENCES
Exact quantitative probabilistic model checking through rational search
TLDR
This work presents a new model checking algorithm that improves the approximate results obtained by scalable iterative techniques to compute exact reachability probabilities, implemented as an extension of the PRISM model checker and evaluated against other exact quantitative model checking engines.
Reachability in MDPs: Refining Convergence of Value Iteration
TLDR
First an interval iteration algorithm is introduced, for which the stopping criterion is straightforward, and the rate of convergence rate is analyzed, which significantly improves the bound on the number of iterations required to get the exact values.
Pareto Curves for Probabilistic Model Checking
TLDR
Dramatic improvements in efficiency on a large set of benchmarks are illustrated and it is shown how the ability to visualise Pareto curves significantly enhances the quality of results obtained from current probabilistic verification tools.
Efficient computation of exact solutions for quantitative model checking
TLDR
This paper proposes a method for obtaining exact results starting from an approximated solution in finite-precision arithmetic, and shows how to obtain a corresponding basis in a linear-programming problem in such a way that the basis is optimal whenever the scheduler attains the worst-case probability.
Safe On-The-Fly Steady-State Detection for Time-Bounded Reachability
  • J. Katoen, Ivan S. Zapreev
  • Computer Science
    Third International Conference on the Quantitative Evaluation of Systems - (QEST'06)
  • 2006
TLDR
A precise procedure for steady-state detection for time-bounded reachability for forward and backward reachability algorithms is obtained and the impact of these results in probabilistic model checking is shown.
The Probabilistic Model Checking Landscape*
  • J. Katoen
  • Computer Science
    2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
  • 2016
TLDR
This paper surveys the algorithmic verification of probabilistic models, in particular Probabilistic model checking, and provides an informal account of the main models, the underlying algorithms, applications from reliability and dependability analysis—and beyond—and describes recent developments towards automated parameter synthesis.
Verification of Markov Decision Processes Using Learning Algorithms
TLDR
A general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs) and focuses on probabilistic reachability, which is a core property for verification, and is illustrated through two distinct instantiations.
PRISM 4.0: Verification of Probabilistic Real-Time Systems
TLDR
A major new release of the PRISMprobabilistic model checker is described, adding, in particular, quantitative verification of (priced) probabilistic timed automata.
Ensuring the Reliability of Your Model Checker: Interval Iteration for Markov Decision Processes
TLDR
This paper presents interval iteration techniques for computing expected accumulated weights (or costs), a considerably broader class of properties, and proposes topological interval iteration, which increases efficiency using a model decomposition into strongly connected components.
...
...