# Statistical Model Checking of Black-Box Probabilistic Systems

@inproceedings{Sen2004StatisticalMC, title={Statistical Model Checking of Black-Box Probabilistic Systems}, author={Koushik Sen and Mahesh Viswanathan and Gul A. Agha}, booktitle={CAV}, year={2004} }

We propose a new statistical approach to analyzing stochastic systems against specifications given in a sublogic of continuous stochastic logic (CSL). Unlike past numerical and statistical analysis methods, we assume that the system under investigation is an unknown, deployed black-box that can be passively observed to obtain sample traces, but cannot be controlled. Given a set of executions (obtained by Monte Carlo simulation) and a property, our algorithm checks, based on statistical… Expand

#### 277 Citations

A Survey of Statistical Model Checking

- Computer Science
- ACM Trans. Model. Comput. Simul.
- 2018

SMC provides a more widely applicable and scalable alternative to analysis of properties of stochastic systems using numerical and symbolic methods, while emphasizing current limitations and tradeoffs between precision and scalability. Expand

Statistical Model Checking: An Overview

- Computer Science, Mathematics
- RV
- 2010

This tutorial surveys the statistical approach to model checking, and outlines its main advantages in terms of efficiency, uniformity, and simplicity. Expand

Bayesian statistical model checking with application to Stateflow/Simulink verification

- Computer Science, Mathematics
- Formal Methods Syst. Des.
- 2013

It is proved that Bayesian SMC can make the probability of giving a wrong answer arbitrarily small, which is essential for scaling up to large Stateflow/Simulink models. Expand

Bayesian statistical model checking with application to Simulink/Stateflow verification

- Computer Science
- HSCC '10
- 2010

It is proved that Bayesian SMC can make the probability of giving a wrong answer arbitrarily small, which enables faster verification than state-of-the-art statistical techniques, while retaining the same error bounds. Expand

Statistical Model Checking with Change Detection

- Computer Science
- LNCS Trans. Found. Mastering Chang.
- 2016

An algorithm that can be used to monitor changes in the probability distribution to satisfy a bounded-time property at runtime and is illustrated by using Plasma Lab to verify a Simulink case study modelling a pig shed temperature controller. Expand

Model-Based Testing of Probabilistic Systems

- Computer Science
- FASE
- 2016

This paper provides algorithms to generate, execute and evaluate test cases from a probabilistic requirements model, and connects ioco-theory for model-based testing and statistical hypothesis testing: these algorithms handle the functional aspects, while statistical methods assess if the frequencies observed during test execution correspond to the probabilities specified in the requirements. Expand

Contributions to Statistical Model Checking

- Computer Science
- 2015

This thesis proposes several contributions to increase the efficiency of SMC and to wider its applicability to a larger class of systems and shows how to extend the applicability ofSMC to estimate the probability of rare-events and considers the problem of detecting probability changes at runtime. Expand

Simulation + Hypothesis Testing for Model Checking Probabilistic Systems A Tutorial

- 2009

Quantitative properties of stochastic systems are usually specified in logics that allow one to compare the measure of executions satisfying certain temporal properties with thresholds. The model… Expand

PVeStA: A Parallel Statistical Model Checking and Quantitative Analysis Tool

- Computer Science
- CALCO
- 2011

PVESTA is presented, an extension and parallelization of the VESTA statistical model checking tool, which supports statistical model Checking of probabilistic real-time systems specified as either discrete or continuous Markov Chains; or Probabilistic rewrite theories in Maude. Expand

On Statistical Model Checking of Stochastic Systems

- Computer Science
- CAV
- 2005

A statistical model checking algorithm that also verifies CSL formulas with unbounded untils, based on Monte Carlo simulation of the model and hypothesis testing of the samples, as opposed to sequential hypothesis testing is presented. Expand

#### References

SHOWING 1-10 OF 23 REFERENCES

Numerical vs. Statistical Probabilistic Model Checking: An Empirical Study

- Computer Science
- TACAS
- 2004

This study relies on highly efficient sequential acceptance sampling tests, which enables statistical solution techniques to quickly return a result with some uncertainty in CSL model checking, and proposes a novel combination of the two solution techniques for verifying CSL queries with nested probabilistic operators. Expand

Approximate Probabilistic Model Checking

- Computer Science
- VMCAI
- 2004

An approximation method to verify quantitative properties on discrete Markov chains using a randomized algorithm to approximate the probability that a property expressed by some positive LTL formula is satisfied with high confidence by a probabilistic system. Expand

Probabilistic Verification of Discrete Event Systems Using Acceptance Sampling

- Computer Science
- CAV
- 2002

A model independent procedure for verifying properties of discrete event systems based on Monte Carlo simulation and statistical hypothesis testing that is probabilistic in two senses and carried out in an anytime manner. Expand

Symbolic Model Checking for Probabilistic Processes

- Computer Science
- ICALP
- 1997

A symbolic model checking procedure for Probabilistic Computation Tree Logic PCTL over labelled Markov chains as models is introduced, based on the algorithm used by Hansson and Jonsson [24], and is efficient because it avoids explicit state space construction. Expand

A Markov Chain Model Checker

- Computer Science
- TACAS
- 2000

A prototype model checker for discrete and continuous-time Markov chains, the Erlangen-Twente Markov Chain Checker (E ⊢ MC2), where properties are expressed in appropriate extensions of CTL, is described. Expand

PRISM: Probabilistic Symbolic Model Checker

- Computer Science
- Computer Performance Evaluation / TOOLS
- 2002

PRISM has been successfully used to analyse probabilistic termination, performance, and quality of service properties for a range of systems, including randomized distributed algorithms, manufacturing systems and workstation clusters. Expand

Verifying Quantitative Properties of Continuous Probabilistic Timed Automata

- Computer Science
- CONCUR
- 2000

This work develops a model checking method for continuous probabilistic timed automata, which improves on the previously known techniques in that it allows the verification of quantitative probability bounds, as opposed to qualitative properties which can only refer to bounds of probability 0 or 1. Expand

Model Checking of Probabalistic and Nondeterministic Systems

- Computer Science
- FSTTCS
- 1995

Model-checking algorithms for extensions of pCTL and p CTL* to systems in which the probabilistic behavior coexists with nondeterminism are presented, and it is shown that these algorithms have polynomial-time complexity in the size of the system. Expand

Model-Checking for Probabilistic Real-Time Systems (Extended Abstract)

- Computer Science
- ICALP
- 1991

This paper extends model-checking to stochastic real-time systems, whose behavior depends on probabilistic choice and quantitative time, with a model that can express constraints like “the delay between the request and the response is distributed uniformly between 2 to 4 seconds”. Expand

Approximate Symbolic Model Checking of Continuous-Time Markov Chains

- Computer Science
- CONCUR
- 1999

A symbolic approximate method for solving the integrals using MTDDs (multi-terminal decision diagrams), a generalisation of MTBDDs, suitable for numerical integration using quadrature formulas based on equally-spaced abscissas, like trapezoidal, Simpson and Romberg integration schemes. Expand