# Markov Automata with Multiple Objectives

@article{Quatmann2017MarkovAW, title={Markov Automata with Multiple Objectives}, author={Tim Quatmann and Sebastian Junges and Joost-Pieter Katoen}, journal={ArXiv}, year={2017}, volume={abs/1704.06648} }

Markov automata combine non-determinism, probabilistic branching, and exponentially distributed delays. This compositional variant of continuous-time Markov decision processes is used in reliability engineering, performance evaluation and stochastic scheduling. Their verification so far focused on single objectives such as (timed) reachability, and expected costs. In practice, often the objectives are mutually dependent and the aim is to reveal trade-offs. We present algorithms to analyze…

## 19 Citations

A Modest Approach to Markov Automata

- Computer ScienceACM Trans. Model. Comput. Simul.
- 2021

Extensions to MODEST, an expressive high-level language with roots in process algebra, are presented that allow large Markov automata models to be specified in a succinct, modular way and illustrate the advantages of MODEST over alternative languages.

Multi-objective Optimization of Long-run Average and Total Rewards

- Computer ScienceTACAS
- 2021

This paper presents an efficient procedure for multi-objective model checking of long-run average reward and total reward objectives as well as their combination for Markov automata, a compositional model that captures both traditional Markov decision processes (MDPs) as a continuous-time variant thereof.

A Modest Approach to Modelling and Checking Markov Automata

- Computer ScienceQEST
- 2019

Extensions to the Modest language and the mcsta model checker are presented to describe and analyse Markov automata models and it is shown that mcsta improves the performance and scalability of Markov Automata model checking compared to earlier and alternative tools.

Stochastic Games with Lexicographic Reachability-Safety Objectives

- Computer ScienceCAV
- 2020

An algorithm is presented that computes lexicographically optimal strategies via a reduction to computation of optimal strategies in a sequence of single-objectives games.

Multi-cost Bounded Tradeoff Analysis in MDP

- Computer ScienceJournal of Automated Reasoning
- 2020

The need for more detailed visual presentations of results beyond Pareto curves is discussed and a first visualisation approach that exploits all the available information from the algorithm to support decision makers is presented.

Multi-cost Bounded Reachability in MDP

- Computer ScienceTACAS
- 2018

The need for output beyond Pareto curves is discussed and the available information from the algorithm is exploited to support decision makers and show the algorithm’s scalability.

Evolutionary-Guided Synthesis of Verified Pareto-Optimal MDP Policies

- Computer Science2021 36th IEEE/ACM International Conference on Automated Software Engineering (ASE)
- 2021

This work uses case studies from the service-based systems and robotic control software domains to show that the new MDP policy synthesis approach can handle a wide range of QoS requirement combinations unsupported by current probabilistic model checkers.

The 2019 Comparison of Tools for the Analysis of Quantitative Formal Models - (QComp 2019 Competition Report)

- Computer ScienceTACAS
- 2019

The challenges in setting up a quantitative verification competition are reported, the results of QComp 2019 are presented, the lessons learned are summarised, and an outlook on the features of the next edition ofQComp is provided.

A Modest Markov Automata Tutorial

- Computer ScienceReasoning Web
- 2019

This tutorial paper gives an introduction to the formalism of Markov automata, to practical modelling of MarkOV automata in the Modest language, and to their analysis with the Modest Toolset.

PAC Statistical Model Checking of Mean Payoff in Discrete- and Continuous-Time MDP

- Computer ScienceArXiv
- 2022

This work provides the first algorithm to compute mean payoff probably approximately correctly in unknown MDP; further, it is extended to unknown CTMDP and demonstrates its practical nature by running experiments on standard benchmarks.

## References

SHOWING 1-10 OF 36 REFERENCES

Analysis of Timed and Long-Run Objectives for Markov Automata

- Computer ScienceLog. Methods Comput. Sci.
- 2014

This paper considers the quantitative analysis of MAs and presents the foundations and details of the algorithms and their correctness proofs, driven by the MAPA modelling language for efficiently generating MAs.

Reachability in MDPs: Refining Convergence of Value Iteration

- Computer Science, MathematicsRP
- 2014

First an interval iteration algorithm is introduced, for which the stopping criterion is straightforward, and the rate of convergence rate is analyzed, which significantly improves the bound on the number of iterations required to get the exact values.

Trade-off analysis meets probabilistic model checking

- Computer ScienceCSL-LICS
- 2014

This paper presents the concepts of quantiles, conditional probabilities and expectations as well as objectives on the ratio between accumulated costs and utility, and extends the results towards combined quantile queries, computing conditional probabilities those conditions are expressed as formulas in probabilistic computation tree logic, and the computation of ratios.

Path-Constrained Markov Decision Processes: bridging the gap between probabilistic model-checking and decision-theoretic planning

- Computer ScienceECAI
- 2012

A new theoretical model, named Path-Constrained Markov Decision Processes, is proposed, which allows system designers to directly optimize safe policies in a single design pass, whose possible executions are guaranteed to satisfy some probabilistic constraints on their paths.

Modelling and Analysis of Markov Reward Automata

- Computer ScienceATVA
- 2014

This paper introduces Markov reward automata, an extension of Markov automata that allows the modelling of systems incorporating rewards (or costs) in addition to nondeterminism, discrete probabilistic choice and continuous stochastic timing.

Model checking nondeterministic and randomly timed systems

- Computer Science
- 2010

This thesis considers continuous-time Markov decision processes (CTMDPs), a model which closely entangles stochasticity and nondeterminism and considers a discretization which allows to compute the maximum and minimum probability to enter a set of goal states in a CTMDP within a given time-bound.

Quantitative Multi-objective Verification for Probabilistic Systems

- Computer ScienceTACAS
- 2011

This work proposes and implements an efficient verification framework for analysing multiple quantitative objectives of systems that exhibit both nondeterministic and stochastic behaviour, and presents two distinct applications of it: firstly, controller synthesis subject to multiple quantitative objective verification; and, secondly, quantitative compositional verification.

Pareto Curves for Probabilistic Model Checking

- Computer ScienceATVA
- 2012

Dramatic improvements in efficiency on a large set of benchmarks are illustrated and it is shown how the ability to visualise Pareto curves significantly enhances the quality of results obtained from current probabilistic verification tools.

On Time with Minimal Expected Cost!

- Computer ScienceATVA
- 2014

This paper provides efficient methods for computing reachability strategies that will both ensure worst case time-bounds as well as provide (near-) minimal expected cost.

Long-Run Rewards for Markov Automata

- Computer ScienceTACAS
- 2017

The computation of long-run average rewards, the most classical problem in continuous-time Markov model analysis, is considered and an algorithm based on value iteration is proposed, which improves the state of the art by orders of magnitude.