Software Fault-Freeness and Reliability Predictions

@inproceedings{Strigini2013SoftwareFA,
  title={Software Fault-Freeness and Reliability Predictions},
  author={Lorenzo Strigini and Andrey Povyakalo},
  booktitle={SAFECOMP},
  year={2013}
}
Many software development practices aim at ensuring that software is correct, or fault-free. In safety critical applications, requirements are in terms of probabilities of certain behaviours, e.g. as associated to the Safety Integrity Levels of IEC 61508. The two forms of reasoning — about evidence of correctness and about probabilities of certain failures — are rarely brought together explicitly. The desirability of using claims of correctness has been argued by many authors, but not been… 
Software Diversity as a Measure for Reducing Development Risk
TLDR
It is demonstrated that, despite the difficulty of predicting the actual advantages of specific practices for achieving diversity, the practice of "forcing" diversity by explicitly mandating diverse designs, development processes, etc., for different versions, rather than just ensuring separate development, is robust, in terms of worst-case effects.
The Unnecessity of Assuming Statistically Independent Tests in Bayesian Software Reliability Assessments
TLDR
Conservative Bayesian approaches are derived on a system’s probability of failure on demand, when operational testing reveals no failures, which is demonstrated in the assessment of a nuclear power-plant safety-protection system under varying levels of skepticism about the i.i.d. assumption.
On the probability of perfection of software-based systems
TLDR
This thesis provides 3 parallel sets of (quasi-)perfection models which could be used individually as a conservative end-to-end argument that reasoning from various types of evidence to the reliability of a software-based system.
On Adaptive Sampling-Based Testing for Software Reliability Assessment
  • R. Pietrantuono, S. Russo
  • Computer Science
    2016 IEEE 27th International Symposium on Software Reliability Engineering (ISSRE)
  • 2016
TLDR
This paper presents an adaptive sampling- based testing (AST) strategy for reliability assessment, where adaptiveness is included to uncover residual faults earlier, while various sampling-based techniques are proposed to improve the efficiency by better exploiting the information available to tester.
Conservative claims about the probability of perfection of software-based systems
TLDR
This paper considers the difficult problem of expressing prior beliefs about the probability of failure on demand, and representing these mathematically, and assumes that, although he cannot provide a full probabilistic description of his uncertainty in a single distribution, the assessor can express some precise but partial belief about the unknowns.
Planning the Unplanned Experiment: Assessing the Efficacy of Standards for Safety Critical Software
TLDR
Assessing standards as written and as applied, several candidate definitions for what it means for a standard to 'work,' and key assessment strategies and study techniques and the pros and cons of each are discussed.
Conservative Confidence Bounds in Safety, from Generalised Claims of Improvement & Statistical Evidence
TLDR
This work proposes a formal probabilistic (Bayesian) organisation for “Proven-in-use”, “globally-at-least-equivalent” and “stress-tested” arguments, and demonstrates scenarios in which formalising such arguments substantially increases confidence in the target system.
...
...

References

SHOWING 1-10 OF 24 REFERENCES
Assessing the risk due to software faults: estimates of failure rate versus evidence of perfection
TLDR
This note clarifies the meaning of the two kinds of statements and how they relate to the probability of failure-free operation, and discusses their practical merits, especially for high required reliability or safety.
The Infeasibility of Quantifying the Reliability of Life-Critical Real-Time Software
This work affirms that the quantification of life-critical software reliability is infeasible using statistical methods, whether these methods are applied to standard software or fault-tolerant
Validation of ultrahigh dependability for software-based systems
TLDR
It appears that engineering practice must take into account the fact that no solution exists, at present, for the validation of ultra-high dependability in systems relying on complex software.
On the Use of Testability Measures for Dependability Assessment
TLDR
It is shown that a high testability is not an unconditionally desirable property for a program, and for programs complex enough that they are unlikely to be completely fault free, increasing testability may produce a program which will be less trustworthy, even after successful testing.
Toward a Formalism for Conservative Claims about the Dependability of Software-Based Systems
TLDR
This paper considers a simple case where an expert makes a claim about the probability of failure on demand of a subsystem of a wider system and is able to express his confidence about that claim probabilistically, and provides formal support for such reasoning.
Confidently Assessing a Zero Probability of Software Failure
TLDR
This work refers to software with reliability requirements of this magnitude as ultra-reliable software, where requirements of 10−7 failures per hour of system reliability translate into a probability of failure (pof) of perhaps 10−9 or less for each individual execution of the software.
Process modelling to support dependability arguments
  • R. Bloomfield, S. Guerra
  • Computer Science
    Proceedings International Conference on Dependable Systems and Networks
  • 2002
TLDR
A method for estimating the number of residual faults at the time of release from a "barrier model" of the development process, where in each phase faults are created or detected, is developed.
The Use of Proof in Diversity Arguments
TLDR
It is shown that assessment of the reliability of the overall fault-tolerant system in this case may take advantage of claims for independence that are more plausible than those involved in design diversity.
Modelling the effects of combining diverse software fault removal techniques
TLDR
It is shown that diversity of fault finding procedures is, in a precisely defined way, ‘a good thing’, and should be applied as widely as possible, and the effects upon reliability of repeated applications of a particular fault finding procedure are not statistically independent.
An empirical comparison of software fault tolerance and fault elimination
  • T. Shimeall, N. Leveson
  • Computer Science
    [1988] Proceedings. Second Workshop on Software Testing, Verification, and Analysis
  • 1988
TLDR
A large-scale experiment comparing software fault tolerance and software fault elimination as approaches to improving software reliability is described, and it was found that n-version programming did not tolerate most of the faults detected by the fault elimination techniques.
...
...