# Sound Value Iteration

@inproceedings{Quatmann2018SoundVI, title={Sound Value Iteration}, author={Tim Quatmann and Joost-Pieter Katoen}, booktitle={CAV}, year={2018} }

Computing reachability probabilities is at the heart of probabilistic model checking. All model checkers compute these probabilities in an iterative fashion using value iteration. This technique approximates a fixed point from below by determining reachability probabilities for an increasing number of steps. To avoid results that are significantly off, variants have recently been proposed that converge from both below and above. These procedures require starting values for both sides. We…

## 29 Citations

Optimistic Value Iteration

- Computer ScienceCAV
- 2020

This paper obtains a lower bound via standard value iteration, uses the result to “guess” an upper bound, and proves the latter’s correctness, and presents this optimistic value iteration approach for computing reachability probabilities as well as expected rewards.

Symblicit exploration and elimination for probabilistic model checking

- Computer ScienceSAC
- 2021

This paper presents a new "symblicit" approach to checking Markov chains and related probabilistic models: a decision diagram that symbolically collects all reachable states and their predecessors and eliminates few explicit states from the explicit state space in a way that preserves all relevant probabilities and rewards.

Correct Probabilistic Model Checking with Floating-Point Arithmetic

- Computer ScienceTACAS
- 2022

This paper shows how to implement fast and correct probabilistic model checking by exploiting the ability of current hardware to control the direction of rounding in floating-point calculations, and outlines the complications in achieving correct rounding from higherlevel programming languages.

Accelerating Interval Iteration for Expected Rewards in Markov Decision Processes

- Computer ScienceICSOFT
- 2020

This paper focuses on the computation of the expected rewards of models and proposes two heuristics to improve the performance of the interval iteration method and proposes a criterion for the correctness of the approximated upper bound.

Stopping Criteria for Value and Strategy Iteration on Concurrent Stochastic Reachability Games

- Mathematics, Computer ScienceArXiv
- 2019

This work considers concurrent stochastic games played on graphs with reachability and safety objectives, and provides the first (anytime) algorithms with stopping criteria for value iteration and strategy iteration.

A Modest Approach to Modelling and Checking Markov Automata

- Computer ScienceQEST
- 2019

Extensions to the Modest language and the mcsta model checker are presented to describe and analyse Markov automata models and it is shown that mcsta improves the performance and scalability of Markov Automata model checking compared to earlier and alternative tools.

Verification of Multiplayer Stochastic Games via Abstract Dependency Graphs

- Computer ScienceLOPSTR
- 2020

An efficient model checking algorithm for alternating-time temporal logic (ATL) on turn-based multiplayer stochastic games with weighted transitions and provides an efficient reduction of the model checking problem to finding the minimum fixed-point assignment on an ADG over the domain of unit intervals extended with certain-zero optimization.

A Modest Approach to Markov Automata

- Computer ScienceACM Trans. Model. Comput. Simul.
- 2021

Extensions to MODEST, an expressive high-level language with roots in process algebra, are presented that allow large Markov automata models to be specified in a succinct, modular way and illustrate the advantages of MODEST over alternative languages.

The Probabilistic Model Checker Storm

- Computer ScienceInternational Journal on Software Tools for Technology Transfer
- 2021

The main features of Storm are reported and how to effectively use them are explained and an empirical evaluation of different configurations of Storm on the QComp 2019 benchmark set is presented.

On Correctness, Precision, and Performance in Quantitative Verification - QComp 2020 Competition Report

- Computer ScienceISoLA
- 2020

This paper surveys the precision guarantees—ranging from exact rational results to statistical confidence statements—offered by the nine participating tools and reports on the experimental evaluation of these trade-offs performed in QComp 2020: the second friendly competition of tools for the analysis of quantitative formal models.

## References

SHOWING 1-10 OF 25 REFERENCES

Exact quantitative probabilistic model checking through rational search

- Computer Science, Mathematics2017 Formal Methods in Computer Aided Design (FMCAD)
- 2017

This work presents a new model checking algorithm that improves the approximate results obtained by scalable iterative techniques to compute exact reachability probabilities, implemented as an extension of the PRISM model checker and evaluated against other exact quantitative model checking engines.

Reachability in MDPs: Refining Convergence of Value Iteration

- Computer Science, MathematicsRP
- 2014

First an interval iteration algorithm is introduced, for which the stopping criterion is straightforward, and the rate of convergence rate is analyzed, which significantly improves the bound on the number of iterations required to get the exact values.

Pareto Curves for Probabilistic Model Checking

- Computer ScienceATVA
- 2012

Dramatic improvements in efficiency on a large set of benchmarks are illustrated and it is shown how the ability to visualise Pareto curves significantly enhances the quality of results obtained from current probabilistic verification tools.

Efficient computation of exact solutions for quantitative model checking

- Computer Science, MathematicsQAPL
- 2012

This paper proposes a method for obtaining exact results starting from an approximated solution in finite-precision arithmetic, and shows how to obtain a corresponding basis in a linear-programming problem in such a way that the basis is optimal whenever the scheduler attains the worst-case probability.

Safe On-The-Fly Steady-State Detection for Time-Bounded Reachability

- Computer ScienceThird International Conference on the Quantitative Evaluation of Systems - (QEST'06)
- 2006

A precise procedure for steady-state detection for time-bounded reachability for forward and backward reachability algorithms is obtained and the impact of these results in probabilistic model checking is shown.

The Probabilistic Model Checking Landscape*

- Computer Science2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
- 2016

This paper surveys the algorithmic verification of probabilistic models, in particular Probabilistic model checking, and provides an informal account of the main models, the underlying algorithms, applications from reliability and dependability analysis—and beyond—and describes recent developments towards automated parameter synthesis.

Verification of Markov Decision Processes Using Learning Algorithms

- Computer ScienceATVA
- 2014

A general framework for applying machine-learning algorithms to the verification of Markov decision processes (MDPs) and focuses on probabilistic reachability, which is a core property for verification, and is illustrated through two distinct instantiations.

PRISM 4.0: Verification of Probabilistic Real-Time Systems

- Computer ScienceCAV
- 2011

A major new release of the PRISMprobabilistic model checker is described, adding, in particular, quantitative verification of (priced) probabilistic timed automata.

Ensuring the Reliability of Your Model Checker: Interval Iteration for Markov Decision Processes

- Computer ScienceCAV
- 2017

This paper presents interval iteration techniques for computing expected accumulated weights (or costs), a considerably broader class of properties, and proposes topological interval iteration, which increases efficiency using a model decomposition into strongly connected components.