Formally Reasoning About Quality

  title={Formally Reasoning About Quality},
  author={Shaull Almagor and Udi Boker and Orna Kupferman},
  journal={Journal of the ACM (JACM)},
  pages={1 - 56}
In recent years, there has been a growing need and interest in formally reasoning about the quality of software and hardware systems. As opposed to traditional verification, in which one considers the question of whether a system satisfies a given specification or not, reasoning about quality addresses the question of how well the system satisfies the specification. We distinguish between two approaches to specifying quality. The first, propositional quality, extends the specification formalism… 

Tables from this paper

Reasoning about Quality and Fuzziness of Strategic Behaviours
This work introduces and study SL[F]—a quantitative ex-tension of SL(Strategy Logic), one of the most natural and expressive logics describing strategic behaviours, and provides a model-checking algorithm based on a quantitative extension of Quantified CTL*.
Being Correct Is Not Enough: Efficient Verification Using Robust Linear Temporal Logic
The logic rLTL is presented and studied, which provides a means to formally reason about both correctness and robustness in system design and is closer to the LTL bound of O(2|φ |).
Reactive synthesis with maximum realizability of linear temporal logic specifications
This work introduces a value function that, by exploiting the LTL semantics, quantifies the level of violation of properties and proposes a novel maximum satisfiability encoding of the search for an optimal implementation.
Directed Control of Discrete Event Systems with $\mathbf{LTL} [\mathcal{F}]$ Specifications
This paper describes control specifications by extended linear temporal logic formulas with propositional quality operators, which are evaluated with real numbers between 0 and 1, to consider how better the specification is satisfied.
Quantitative Assume Guarantee Synthesis
Researchers have identified the \(\text {GR(1)}\) fragment of LTL, which supports assume-guarantee reasoning and for which synthesis has an efficient symbolic solution.
Finite LTL Synthesis with Environment Assumptions and Quality Measures
A natural variant of the finite LTL synthesis problem in which strategy guarantees are predicated on specified environment behavior is studied, and a quantitative extension of LTL is explored that supports specification of quality measures, utilizing it to synthesize high-quality strategies.
Latticed-LTL synthesis in the presence of noisy inputs
The noisy-synthesis problem for LLTL is 2EXPTIME-complete, as is traditional LTL synthesis, and a new type of incomplete information is introduced, where the truth values of some of the input signals may be noisy, and the transducer should still realize ψ in the desired satisfaction value.
Maximum Realizability for Linear Temporal Logic Specifications
This work provides a quantitative semantics of sets of safety specifications, and proposes an algorithm for synthesizing implementations that are optimal with respect to this quantitative semantics, and develops a MaxSAT encoding which allows for maximizing the quantitative satisfaction of the safety specifications.
Synthesis for Multi-weighted Games with Branching-Time Winning Conditions
This work encodes both problems to the calculation of maximal fixed points on dependency graphs, thus achieving on-the-fly algorithms with the possibility of early termination in both the model checking problem and the synthesis problem.


Formalizing and Reasoning about Quality
By extending the automata-theoretic approach for LTL to a setting that takes quality into an account, it is able to solve the above problems and show that reasoning about LTL has roughly the same complexity as reasoning about traditional LTL.
Discounting in LTL
One direction in this effort is to refine the “eventually” operators of temporal logic to discounting operators: the satisfaction value of a specification is a value in [0,1], where the longer it takes to fulfill eventuality requirements, the smaller the satisfactionvalue is.
Model checking discounted temporal properties
Sanity Checks in Formal Verification
Examination of existing work on vacuity and coverage argues that, in many aspects, the two checks are essentially the same: both are based on repeating the verification process on some mutant input, whereas in coverage, mutations are in the system.
Model checking
Model checking tools, created by both academic and industrial teams, have resulted in an entirely novel approach to verification and test case generation that often enables engineers in the electronics industry to design complex systems with considerable assurance regarding the correctness of their initial designs.
Better Quality in Synthesis through Quantitative Objectives
It is shown how automata with lexicographic mean-payoff conditions can be used to express many interesting quantitative properties for reactive systems, and how quantitative properties to measure the "goodness" of an implementation are used.
Temporal Specifications with Accumulative Values
This work shows that extending the fragment of CTL that has only the EX, EF, AX, and AG temporal modalities by prefix-accumulation assertions and extending LTL with path-accUMulation assertions, result in temporal logics whose model-checking problem is decidable.
High-Quality Synthesis Against Stochastic Environments
The stochastic setting is introduced, where the goal is to generate a transducer that maximizes the expected quality of a computation, subject to a given distribution of the input signals.
Modalities for Model Checking: Branching Time Logic Strikes Back
Quantitative verification: models, techniques and tools
The modelling approach is described and the usefulness of the methodology illustrated with an example of a real-world protocol standard - Bluetooth device discovery - that has been analysed using the PRISM model checker.