Accuracy of Software Reliability Prediction from Different Approaches
The conventional definition of reliability (and reliability engineering as discipline) suggests that reliability is a quantifiable performance requirement of a product or system. This means that reliability can be specified, designed for, predicted, measured and demonstrated, using applicable mathematical and statistical models. This paper argues against the futile and erroneous quantification of reliability. Many reliability engineering practices are incorrect, misleading, fraudulent or simply a waste of time and money. Particular attention is paid to the methods for reliability prediction and reliability testing. Common sense suggests that reliability is the absence of failures in products and systems, and that reliability is achieved by preventing the creation of failures. This definition excludes the quantification of reliability, and is consistent with the Total Quality Management principle of continuous improvement in design and production. Reliability definitions In reaction to perceived low reliability of military equipment at the time, the US Department of Defense has introduced a series of military standards and specifications on quality and reliability since the 1950s. The quality standard was Mil-Q-9858, which contained requirements for a quality management system for suppliers, which would be audited by DoD quality staff. It was thought that this approach would provide greater assurance of quality. The well-known (but also controversial) ISO9000 standard is the direct descendant of Mil-Q-9858, the only significant difference being third party assessment (O’Connor 2001). In reliability engineering, standards and handbooks were produced for reliability programme management (Mil-Std-785), reliability prediction (Mil-Hdbk-217) and reliability demonstration (Mil-Std-781). These standards gained considerable acceptance in the western world, not only in the defence industry, but also in other industries such as nuclear, commercial aircraft, automotive, telecommunications, etc. These standards and handbooks address reliability as quantifiable performance requirement, and prescribe methods to be used to achieve and demonstrate reliability in products and systems. In addition to these standards and handbooks, a large number of textbooks on reliability have been published.