Feedback-Directed Random Test Generation

  title={Feedback-Directed Random Test Generation},
  author={Carlos Pacheco and Shuvendu K. Lahiri and Michael D. Ernst and Thomas Ball},
  journal={29th International Conference on Software Engineering (ICSE'07)},
We present a technique that improves random test generation by incorporating feedback obtained from executing test inputs as they are created. Our technique builds inputs incrementally by randomly selecting a method call to apply and finding arguments from among previously-constructed inputs. As soon as an input is built, it is executed and checked against a set of contracts and filters. The result of the execution determines whether the input is redundant, illegal, contract-violating, or… 

Figures from this paper

Directed random testing
Directed random testing is a new approach to test generation that overcomes limitations, by combining a bottom-up generation of tests with runtime guidance and reveals more errors and achieves higher code coverage.
Dependence-Guided Random Test Generation
This paper proposes a novel dependence-guided enhancement to the random test generation approach that exploits how methods are implemented rather than how they are used, and thus does not depend on a client code.
Feedback-controlled random test generation
This paper proposes a method named feedback-controlled random test generation, which aggressively controls the feedback in order to promote diversity of generated tests and increases branch coverage on large-scale utility libraries.
A uniform random test data generator for path testing
Finding the needles in the haystack: Generating legal test inputs for object-oriented programs
A scalable technique that combines dynamic analysis with random testing to help an input generator create legal test inputs without a formal specification, even for programs in which most sequences are illegal.
Cooperative verifier-based testing with CoVeriTest
CoVeriTest is presented, a hybrid technique for test-suite generation that iteratively applies different conditional model checkers and allows users to adjust the level of cooperation and to configure individual time limits for each conditional modelChecker.
Semi-automatic Search-Based Test Generation
  • Y. Pavlov, G. Fraser
  • Computer Science
    2012 IEEE Fifth International Conference on Software Testing, Verification and Validation
  • 2012
A semi-automatic test generation approach based on the search-based EVO SUITE tool is presented, and the usefulness and potential on a set of example classes are evaluated.
Diversity-based web test generation
This work presents a novel web test generation algorithm that pre-selects the most promising candidate test cases based on their diversity from previously generated tests, and achieves higher coverage and fault detection rates significantly earlier than crawling-based and search-based web test generators.
Developing and Evaluating Objective Termination Criteria for Random Testing
This article proposes two promising termination criteria, “All Equivalent” (AEQ) and "All Included in One" (AIO), applicable to random testing, and model and implement them in the context of a general random testing process composed of independent random testing sessions.
Isomorphic regression testing: executing uncovered branches without test augmentation
This paper makes the first implementation of isomorphic regression testing through an approach named ISON, which creates program variants by negating branch conditions and detects a number of faults not detected by a popular automated test generation tool under the scenario of regression testing.


DART: directed automated random testing
DART is a new tool for automatically testing software that combines three main techniques, automated extraction of the interface of a program with its external environment using static source-code parsing, and dynamic analysis of how the program behaves under random testing and automatic generation of new test inputs to direct systematically the execution along alternative program paths.
Eclat: Automatic Generation and Classification of Test Inputs
A technique that selects, from a large set of test inputs, a small subset likely to reveal faults in the software under test, which is seen as an error-detection technique and implemented in the Eclat tool.
Mutation 2000: uniting the orthogonal
This paper outlines a design for a system that will approximate mutation, but in a way that will be accessible to every day programmers, and believes this system could be efficient enough to be adopted by leading-edge software developers.
Random testing is performed where the defects are NOT identified in regular intervals and random input is used to test the system's reliability and performance to save time and effort.
An Evaluation of Exhaustive Testing for Data Structures
It is indicated that it is feasible to use exhaustive testing to obtain, within a reasonable amount of time, a high-quality test suite that can detect almost all faults and achieve complete code and specification coverage.
An Empirical Comparison of Automated Generation and Classification Techniques for Object-Oriented Unit Testing
A novel pair, model-based symbolic testing is developed, and the results show that the pairs are complementary (i.e., reveal faults differently), with their respective strengths and weaknesses.
An Evaluation of Random Testing
Simulation results are presented which suggest that random testing may often be more cost effective than partition testing schemes and results of actual random testing experiments are presented, which confirm the viability of random testing as a useful validation tool.
Test input generation for java containers using state matching
Techniques for automated test input generation of Java container classes, using state matching based techniques and random selection for generating test inputs, are described and abstract matching turns out to be a powerful approach for generatingtest inputs to obtain high predicate coverage.
JCrasher: an automatic robustness tester for Java
JCrasher attempts to detect bugs by causing the program under test to ‘crash’, that is, to throw an undeclared runtime exception, to test the behavior of public methods under random data.
Test input generation with java PathFinder
The main contribution of this work is to show how efficient white-box test input generation can be done for code manipulating complex data, taking into account complex method preconditions.