Statically-directed dynamic automated test generation

@inproceedings{Babic2011StaticallydirectedDA,
  title={Statically-directed dynamic automated test generation},
  author={Domagoj Babic and Lorenzo Martignoni and Stephen McCamant and Dawn Xiaodong Song},
  booktitle={ISSTA '11},
  year={2011}
}
We present a new technique for exploiting static analysis to guide dynamic automated test generation for binary programs, prioritizing the paths to be explored. Our technique is a three-stage process, which alternates dynamic and static analysis. In the first stage, we run dynamic analysis with a small number of seed tests to resolve indirect jumps in the binary code and build a visibly pushdown automaton (VPA) reflecting the global control-flow of the program. Further, we augment the computed… 

Figures and Tables from this paper

Combining static analysis error traces with dynamic symbolic execution (experience paper)
TLDR
This case study can inform static analysis and dynamic symbolic execution tool developers as to where improvements may be necessary, and serve as a call to arms for researchers interested in combining symbolic execution and static analysis to identify more suitable benchmark suites for evaluation of research ideas.
Transparently improving regression testing using symbolic execution
TLDR
Two techniques for amplifying the effect of existing test suites using a lightweight symbolic execution mechanism are presented and katch can automatically synthesise inputs that significantly increase the patch coverage achieved by the existing manual test suites, and find bugs at the moment they are introduced.
Combining static analysis and targeted symbolic execution for scalable bug-finding in application binaries
TLDR
Experiments on real-life programs demonstrate that the tool WatSym, built on selective symbolic execution engine S2E, can generate crashing inputs in feasible time and order of magnitude better than symbolic approaches (as embodied by S1E) failed.
make test-zesti: A symbolic execution solution for improving regression testing
TLDR
This paper presents a technique for amplifying the effect of existing test suites using a lightweight symbolic execution mechanism, which thoroughly checks all sensitive operations executed by the test suite for errors, and explores additional paths around sensitive operations.
Targeted Test Generation for Actor Systems
TLDR
This paper proposes a method to support generation of system-level tests to cover a given code location in an actor system and prune the search space by using two heuristics combined with a feedback-directed technique.
Java Ranger: statically summarizing regions for efficient symbolic execution of Java
Merging execution paths is a powerful technique for reducing path explosion in symbolic execution. One approach, introduced and dubbed “veritesting” by Avgerinos et al., works by translating abounded
adapting software testing techniques to enhance software security
DOWSER is a ‘guided’ fuzzer that combines taint tracking, program analysis and symbolic execution to find buffer overflow and underflow vulnerabilities buried deep in a program’s logic. The key idea
Chopped Symbolic Execution
TLDR
This paper proposes chopped symbolic execution, a novel form of symbolic execution that allows users to specify uninter-esting parts of the code to exclude during the analysis, thus only targeting the exploration to paths of importance.
DASE: Document-Assisted Symbolic Execution for Improving Automated Test Generation
TLDR
DASE guides symbolic execution to focus on those inputs that are semantically more important, and increases line coverage, branch coverage, and call coverage by 5.27–22.10%, 5.83–21.25% and 2.43% respectively.
...
...

References

SHOWING 1-10 OF 56 REFERENCES
Heuristics for Scalable Dynamic Test Generation
  • Jacob Burnim, Koushik Sen
  • Computer Science
    2008 23rd IEEE/ACM International Conference on Automated Software Engineering
  • 2008
TLDR
Several heuristic search strategies are presented, including a novel strategy guided by the control flow graph of the program under test, which achieves significantly greater branch coverage on the same testing budget than concolic testing with a traditional depth-first search strategy.
DART: directed automated random testing
TLDR
DART is a new tool for automatically testing software that combines three main techniques, automated extraction of the interface of a program with its external environment using static source-code parsing, and dynamic analysis of how the program behaves under random testing and automatic generation of new test inputs to direct systematically the execution along alternative program paths.
Path Optimization in Programs and Its Application to Debugging
TLDR
An efficient algorithm for path optimization based on weighted pushdown systems is derived and a shortest program path is found that touches these predictors, which can be used by a programmer to debug his software.
Loop-extended symbolic execution on binary programs
Mixed concrete and symbolic execution is an important technique for finding and understanding software bugs, including security-relevant ones. However, existing symbolic execution techniques are
DSD-Crasher: A hybrid analysis tool for bug finding
TLDR
DSD-Crasher is a bug finding tool that follows a three-step approach to program analysis that yields benefits compared to past two-step combinations in the literature, and in the evaluation with third-party applications, demonstrates higher precision over tools that lack a dynamic step and higher efficiency over tools That lack a static step.
Execution synthesis: a technique for automated software debugging
TLDR
ESD--a debugger based on execution synthesis--is evaluated on popular software and reproduces on its own several real concurrency and memory safety bugs in less than three minutes, thus incurring no runtime overhead and being practical for use in production systems.
Subsumer-First: Steering Symbolic Reachability Analysis
TLDR
Subsumer-first is presented, a new approach for steering symbolic reachability analysis that targets both minimal counterexample discovery and efficiency of exhaustive exploration and can be applied to improve efficiency of software verification tools based on predicate abstraction.
KLEE: Unassisted and Automatic Generation of High-Coverage Tests for Complex Systems Programs
TLDR
A new symbolic execution tool, KLEE, capable of automatically generating tests that achieve high coverage on a diverse set of complex and environmentally-intensive programs, and significantly beat the coverage of the developers' own hand-written test suite is presented.
EXE: Automatically Generating Inputs of Death
TLDR
This article presents EXE, an effective bug-finding tool that automatically generates inputs that crash real code by solving the current path constraints to find concrete values using its own co-designed constraint solver, STP.
Directed explicit-state model checking in the validation of communication protocols
TLDR
The class of A* directed search algorithms are presented and proposed heuristics together with bitstate compression techniques for the search of safety property violations and great reductions in the length of the error trails are achieved.
...
...