Stochastic Games with Lexicographic Reachability-Safety Objectives

  title={Stochastic Games with Lexicographic Reachability-Safety Objectives},
  author={Krishnendu Chatterjee and Joost-Pieter Katoen and Maximilian Weininger and Tobias Winkler},
  journal={Computer Aided Verification},
  pages={398 - 420}
We study turn-based stochastic zero-sum games with lexicographic preferences over reachability and safety objectives. Stochastic games are standard models in control, verification, and synthesis of stochastic reactive systems that exhibit both randomness as well as angelic and demonic non-determinism. Lexicographic order allows to consider multiple objectives with a strict preference order over the satisfaction of the objectives. To the best of our knowledge, stochastic games with lexicographic… 

Stochastic Games with Disjunctions of Multiple Objectives (Technical Report)

A fine-grained overview of strategy and computational complexity is presented and a novel value iteration-style algorithm for approximating the set of Pareto optimal thresholds for a given DQ is proposed.

Symbolic Verification and Strategy Synthesis for Turn-based Stochastic Games

This paper describes how to build and verify turn-based stochastic games against either zero-sum or Nash equilibrium based temporal logic specifications, and evaluates the performance of the approach, showing that it is superior in a number of cases and that strategies synthesised in a symbolic fashion can be considerably more compact.

Comparison of Algorithms for Simple Stochastic Games (Full Version)

This paper compares the three known classes of algorithms -- value iteration, strategy iteration and quadratic programming -- both theoretically and practically and suggests several improvements for all algorithms, including the first approach based on quadratics programming that avoids transforming the stochastic game to a stopping one.

Arena-Independent Finite-Memory Determinacy in Stochastic Games

These contributions further the understanding of arena-independent finite-memory (AIFM) determinacy, i.e., the study of objectives for which memory is needed, but in a way that only depends on limited parameters of the game graphs.

Entropy-Guided Control Improvisation

This framework, which extends the state-of-the-art by supporting arbitrary combinations of adversarial and probabilistic uncertainty in the environment, enables a flexible modeling formalism which it is argued, theoretically and empirically, remains tractable.

Model-Free Learning of Safe yet Effective Controllers

A model-free reinforcement learning algorithm is proposed that learns a policy that first maximizes the probability of ensuring safety, then the likelihood of satisfying the given LTL specification, and lastly, the sum of discounted Quality of Control rewards.

The surprizing complexity of generalized reachability games

It is proved that deciding the winner in such games is $\PSPACE$-complete, although it is fixed-parameter tractable with the number of reachability objectives as parameter, and it is shown that bounding the size of the reachability sets gives two natural subclasses where deciding thewinner can be done efficiently.

Parameterized complexity of games with monotonically ordered omega-regular objectives

This work investigates in detail turned-based games with monotonically ordered and {\omega}-regular objectives and provides a parametric complexity analysis and shows that the threshold problem is in FPT for all monotonic preorders and all classical types of {\omeGA}- regular objectives.

Markov automata with multiple objectives

Algorithms to analyze several objectives simultaneously and approximate Pareto curves are presented, including, e.g., several (timed) reachability objectives, or various expected cost objectives.

Strategy Synthesis for Stochastic Games with Multiple Long-Run Objectives

This work shows that strategies constructed from Pareto set approximations of expected energy objectives are e-optimal for the corresponding average rewards in turn-based stochastic games whose winning conditions are conjunctions of satisfaction objectives for long-run average rewards.

On Stochastic Games with Multiple Objectives

It is shown that finding a winning strategy is PSPACE-hard in general and undecidable for deterministic strategies, and it is proved that optimal strategies, if they exists, may require infinite memory and randomisation.

Better Quality in Synthesis through Quantitative Objectives

It is shown how automata with lexicographic mean-payoff conditions can be used to express many interesting quantitative properties for reactive systems, and how quantitative properties to measure the "goodness" of an implementation are used.

Optimal cost almost-sure reachability in POMDPs

Quantitative verification and strategy synthesis for stochastic games

Perfect-Information Stochastic Games with Generalized Mean-Payoff Objectives*

  • K. ChatterjeeL. Doyen
  • Computer Science
    2016 31st Annual ACM/IEEE Symposium on Logic in Computer Science (LICS)
  • 2016
The almost-sure problem as well as the general basic decision problem are coNP-complete, significantly improving the previous results and presenting a matching exponential upper bound for the problem.

Multi-Objective Model Checking of Markov Decision Processes

It is shown that one can compute an approximate Pareto curve with respect to a set of ω-regular properties in time polynomial in the size of the MDP.