Unit test virtualization with VMVM

@article{Bell2014UnitTV,
  title={Unit test virtualization with VMVM},
  author={Jonathan Bell and Gail E. Kaiser},
  journal={Proceedings of the 36th International Conference on Software Engineering},
  year={2014}
}
  • Jonathan Bell, G. Kaiser
  • Published 31 May 2014
  • Computer Science
  • Proceedings of the 36th International Conference on Software Engineering
Testing large software packages can become very time intensive. To address this problem, researchers have investigated techniques such as Test Suite Minimization. Test Suite Minimization reduces the number of tests in a suite by removing tests that appear redundant, at the risk of a reduction in fault-finding ability since it can be difficult to identify which tests are truly redundant. We take a completely different approach to solving the same problem of long running test suites by instead… 

Figures and Tables from this paper

VMVM: unit test virtualization for Java
TLDR
This Tool Demonstration presents the implementation of Unit Test Virtualization, VMVM (pronounced "vroom-vroom") and summarizes an evaluation of the implementation on 20 real-world Java applications, showing that it reduces test suite execution time by up to 97% (on average, 62%).
Vroom: Faster Build Processes for Java
TLDR
Two complementary approaches into a system that seamlessly supports Ant and Maven JUnit build processes isolate in-memory dependencies among test cases, which otherwise are isolated inefficiently by restarting the Java Virtual Machine (JVM) before every test.
Debugging the performance of Maven’s test isolation: experience report
TLDR
It was found that different build systems use different mechanisms for communicating between these multiple processes, and that because of this design decision, running tests with some build systems could be faster than with others.
Assessing the state and improving the art of parallel testing for C
TLDR
This work presents an approach to analyze the potential of C test suites for safe concurrent execution, and execute tests concurrently with different parallelization strategies using processes or threads if it is found to be safe, and finds that multi-threading rarely outperforms multi-processing.
Accelerating Regression Testing for Scaled Self-Driving Cars with Lightweight Virtualization -- A Case Study
  • C. Berger
  • Computer Science
    2015 IEEE/ACM 1st International Workshop on Software Engineering for Smart Cyber-Physical Systems
  • 2015
TLDR
In a case study about accelerating regression testing for scaled self-driving cars, the use of namespace-separation based lightweight virtualization that powers solutions like Docker or Google's lmctfy is evaluated and the total test execution time is reduced.
Fine-Grained Test Minimization
TLDR
This work proposes a novel approach for fine-grained test case minimization, based on the inference of a test suite model that enables automated test reorganization within test cases, that enables removing redundancies at the test statement level, while preserving the coverage and test assertions of the test suite.
Efficient dependency detection for safe Java test acceleration
TLDR
A novel approach to detecting all dependencies between test cases in large projects that can enable safe exploitation of parallelism and test selection with a modest analysis cost is presented.
FastFI: Accelerating Software Fault Injections
TLDR
FASTFI is proposed, which takes patch files that specify source code mutations as an input, conducts an automated source code analysis to identify the function they target, and then automatically parallelizes the execution of all mutants that target the same function.
Techniques for Evolution-Aware Runtime Verification
TLDR
This work presents the first evolution-aware RV techniques that reduce RV overhead across multiple program versions, and compares their techniques with the evolution-unaware (base) RV when monitoring test executions in 200 versions of 10 open-source projects.
Fast Kernel Error Propagation Analysis in Virtualized Environments
TLDR
This paper presents a novel approach that leverages static and dynamic analysis alongside modern operating system and virtual machine features to reduce SFI test latencies for operating system kernel components while enabling efficient and accurate detection of internal state corruptions.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 49 REFERENCES
VMVM: unit test virtualization for Java
TLDR
This Tool Demonstration presents the implementation of Unit Test Virtualization, VMVM (pronounced "vroom-vroom") and summarizes an evaluation of the implementation on 20 real-world Java applications, showing that it reduces test suite execution time by up to 97% (on average, 62%).
An Empirical Study of JUnit Test-Suite Reduction
TLDR
The results show that the four traditional test-suite reduction techniques can effectively reduce these JUnit test suites without substantially reducing their fault-detection capability.
Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction
TLDR
This work presents a new approach for test suite reduction that attempts to use additional coverage information of test cases to selectively keep some additional test cases in the reduced suites that are redundant with respect to the testing criteria used for suite minimization, with the goal of improving the FDE retention of the reduction suites.
MINTS: A general framework and tool for supporting test-suite minimization
  • Hwa-You Hsu, A. Orso
  • Computer Science
    2009 IEEE 31st International Conference on Software Engineering
  • 2009
TLDR
This paper proposes a test-suite minimization framework that is freely-available and can be interfaced with a number of different state-of-the-art solvers, and implements this framework in a tool, called MINTS, that can be used to instantiate anumber of different test- Suites minimization problems and efficiently find an optimal solution for such problems using different solvers.
On-demand test suite reduction
TLDR
An on-demand test suite reduction approach that attempts to select a representative subset satisfying the same test requirements as an initial test suite conceding at most l% loss in fault-detection capability for at least c% of the instances in which it is applied.
An empirical study of the effects of minimization on the fault detection capabilities of test suites
TLDR
An experiment in which the costs and benefits of minimizing test suites of various sizes for several programs are compared reveals that the fault detection capabilities of test suites can be severely compromised by minimization.
Empirical studies of test case prioritization in a JUnit testing environment
TLDR
The analyses show that test case prioritization can significantly improve the rate of fault detection of JUnit test suites, but also reveal differences with respect to previous studies that can be related to the language and testing paradigm.
Finding bugs by isolating unit tests
TLDR
Rather than just executing entire test suites within frameworks such as JUnit, this work executes each test in isolation, thus removing masking effects that might be present in the test suites, and hypothesize that this can reveal previously hidden dependencies between program units or tests.
A study of effective regression testing in practice
TLDR
This proposed hybrid technique combines modification, minimization and prioritization-based selection using a list of source code changes and the execution traces from test cases run on previous versions to identify a representative subset of all test cases that may result in different output behavior on the new software version.
A methodology for controlling the size of a test suite
TLDR
The authors present a technique that selects from a test suite a representative set of test cases that provides the same measure of coverage as the test suite by means of the data flow testing methodology.
...
1
2
3
4
5
...