Integrated Environment for Diagnosing Verification Errors
The simple and often imprecise specifications that programmers may write are a significant limit to a wider application of rigorous program verification techniques. Part of the reason why non-specialists find writing good specification hard is that, when verification fails, they receive little guidance as to what the causes might be, such as implementation errors or inaccurate specifications. To address these limitations, this paper presents two-step verification, a technique that combines implicit specifications, inlining, and loop unrolling to provide improved user feedback when verification fails. Two-step verification performs two independent verification attempts for each program element: one using standard modular reasoning, and another one after inlining and unrolling; comparing the outcomes of the two steps suggests which elements should be improved. Two-step verification is implemented in AutoProof, our static verifier for Eiffel programs integrated in EVE (the Eiffel Verification Environment) and available online. 1 The Trouble with Specs There was a time when formal verification required heroic efforts and was the exclusive domain of a small group of visionaries. That time is now over; but formal techniques still seem a long way from becoming commonplace. If formal verification techniques are to become a standard part of the software development process—and they are—we have to understand and remove the obstacles that still prevent non-specialists from using them. A crucial issue is specification. Program correctness is a relative notion, in that a program is correct not in absolute terms but only relative to a given specification of its expected behavior; in other words, verified programs are only as good as their specification. Unfortunately, many programmers are averse to writing specifications, especially formal ones, for a variety of reasons that mostly boil down a benefit-to-effort ratio perceived as too low. Writing formal specifications may require specialized skills and experience; and the concrete benefits are dubious in environments that value productivity, assessed through simple quantitative measures, more than quality. Why should programmers subject themselves to the taxing exercise of writing specifications in addition to implementations, if there is not much in it for them other than duplicated work? There are, however, ways to overcome these obstacles. First, not all program verification requires providing a specification because specifications can sometimes be inferred from the program text [8,24,18] or from observing common usage patterns [14,37,36]. In particular, some useful specifications are implicit in the programming language semantics: types must be compatible; array accesses must be within bound; dereferenced pointers must be non-null; arithmetic operations must not overflow; and so on. Second, programmers are not incorrigibly disinclined to write specifications [7,15,31], provided it does not require subverting their standard programming practices, produces valuable feedback, and brings tangible benefits. Interesting challenges lie in reducing the remaining gap between the user experience most programmer are expecting and the state-ofthe-art of formal verification techniques and tools. In this paper, we combine a series of techniques to improve the applicability and usability of static program checkers such as Spec# , Dafny , or one of the incarnations of ESC/Java [17,11,20], which verify functional properties of sequential programs specified using contracts (preconditions, postconditions, and class invariants). To enable verification of code with little or no specification, we deploy implicit contracts, routine inlining, and loop unrolling. Implicit contracts (described in Section 3) are simple contracts that follow from the application of certain programming constructs; for example, every array access implicitly requires that the index be within bounds. Routine inlining (Section 4) replaces calls to routines with the routines’ bodies, to obviate the need for a sufficiently expressive callee’s specification when reasoning in the caller’s context. Similarly, loop unrolling makes it possible to reason about loops with incomplete or missing invariants by directly considering the concrete loop bodies. Implicit contracts, inlining, and unrolling—besides being directly useful to improve reasoning with scarce specification—are the ingredients of two-step verification (Section 5), a technique to improve the usability and effectiveness of static checking. Two-step verification performs two verification attempts for every routine: the first one is a standard static checking, using modular reasoning based on programmer-written contracts (whatever they are) plus possibly implicit contracts; the second one uses inlining and unrolling. Comparing the outcomes of the two verification steps provides valuable information about the state of the program and its specification, which is then used to improve the feedback given to users. Bugs violating implicit contracts make for early error detection, and may convince users to add some explicit contracts that avoid them. Discrepancies between the verification of calls with and without inlining may help understand whether failed verification attempts are due to errors in the implementation or in the specification, or simply a more accurate specification is required. For example, a call to some routine r that may violate r’s user-written precondition but verifies correctly after inlining r’s body signals that r’s precondition may be unnecessarily strong. Two-step verification is applied completely automatically: users get the most complete feedback based on the integration of the results of the various verification steps—with and without inlining and similar techniques.