Optimizing Test Placement for Module-Level Regression Testing

@article{Shi2017OptimizingTP,
  title={Optimizing Test Placement for Module-Level Regression Testing},
  author={August Shi and Suresh Thummalapenta and Shuvendu K. Lahiri and Nikolaj Bj{\o}rner and Jacek Czerwonka},
  journal={2017 IEEE/ACM 39th International Conference on Software Engineering (ICSE)},
  year={2017},
  pages={689-699}
}
Modern build systems help increase developer productivityby performing incremental building and testing. Thesebuild systems view a software project as a group of interdependentmodules and perform regression test selection at themodule level. However, many large software projects have imprecisedependency graphs that lead to wasteful test executions. Ifa test belongs to a module that has more dependencies than theactual dependencies of the test, then it is executed unnecessarilywhenever a code… 

Figures and Tables from this paper

File-level vs. module-level regression test selection for .NET
TLDR
This work describes the key challenges of bringing the Ekstazi idea to the .NET platform, and compares it to an incremental build system, which, out of the box, provides module-level dependency tracking and skipping tasks whenever dependencies of a task do not change between the current and the last successful build.
Machine Learning to Uncover Correlations Between Software Code Changes and Test Results
TLDR
The goal of this thesis is to create a test execution model which uses supervised machine learning techniques to predict potential points of failure in a set of tests to reduce the number of test cases needed to be executed in order to test changes in code.
Understanding and Improving Regression Test Selection in Continuous Integration
TLDR
Comparison of module-and class-level RTS techniques in a cloud-based CI environment, Travis and results show that RTS can be beneficial for the developers in theCI environment, and RTS not only saves time but also avoids misleading developers by flaky test failures.
Root causing flaky tests in a large-scale industrial setting
TLDR
This work describes an end-to-end framework that helps identify flaky tests and understand their root causes, and uses a preliminary tool, called RootFinder, to find differences in the logs of passing and failing runs.
Method-Level Test Selection for Continuous Integration with Static Dependencies and Dynamic Execution Rules
TLDR
This work proposes a Method-level tEst SelecTion approach (i.e., MEST), which utilizes static dependencies and dynamic execution rules and can further speed up the feedback of CI testing and improve the fault detection efficiency ofCI testing.
DeFlaker: Automatically Detecting Flaky Tests
TLDR
This work presents the first extensive evaluation of rerunning failing tests and proposes a new technique, called DeFlaker, that detects if a test failure is due to a flaky test without rerunning and with very low runtime overhead.
Test case prioritization using build history and test distances: an approach for improving automotive regression testing in continuous integration environments
TLDR
This dissertation aims to shorten the build cycle in CI environments by detecting regression faults earlier, allowing software developers to integrate and verify their changes more frequently and continuously.
Detecting and Fixing Flaky Tests
TLDR
Techniques and tools for making regression testing more reliable by detecting flaky tests and mitigating their effects and improving testing of code for computational science, where reproducibility is currently a big challenge that can be alleviated by software testing techniques are developed.
A Cost-efficient Approach to Building in Continuous Integration
TLDR
This paper proposes SmartBuildSkip, a technique which predicts the first builds in a sequence of build failures and the remaining build failures separately, and is customizable, allowing developers to select different preferred trade-offs of saving many builds vs. observing build failures early.
...
1
2
3
...

References

SHOWING 1-10 OF 23 REFERENCES
Techniques for improving regression testing in continuous integration development environments
TLDR
This work presents algorithms that make continuous integration processes more cost-effective, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost- effective in this context.
Practical regression test selection with dynamic file dependencies
TLDR
A new, lightweight RTS technique, called Ekstazi, is proposed that can integrate well with testing frameworks and has lower end-to-end time than the existing techniques, despite the fact that it selects more tests.
A safe, efficient regression test selection technique
TLDR
Initial empirical studies indicate that the technique can significantly reduce the cost of regression testing modified software and is at lease as precise as other safe regression test selection algorithms.
Scaling regression testing to large software systems
TLDR
A new regression-test-selection technique for Java programs that is safe, precise, and yet scales to large systems is presented that can efficiently reduce the regression testing effort and, thus, achieve considerable savings.
The evolution of Java build systems
TLDR
The evolution of build systems based on two popular Java build languages (i.e., ANT and Maven) are studied from two perspectives: a static perspective, where the complexity of build system specifications are examined using software metrics adopted from the source code domain; and a dynamic perspective,where the complexity and coverage of representative build runs are measured.
A Tool for Optimizing the Build Performance of Large Software Code Bases
  • A. Telea, L. Voinea
  • Computer Science
    2008 12th European Conference on Software Maintenance and Reengineering
  • 2008
TLDR
Build Analyzer is presented, a tool that helps developers optimize the build performance of huge systems written in C and shows why a build is costly, how its cost is spread over the entire code base, which headers cause build bottlenecks, and suggests ways to refactor these to reduce the cost.
Regression testing minimization, selection and prioritization: a survey
TLDR
This paper surveys each area of minimization, selection and prioritization technique and discusses open problems and potential directions for future research.
An empirical study of build system migrations in practice: Case studies on KDE and the Linux kernel
As the build system, i.e. the infrastructure that constructs executable deliverables out of source code and other resources, tries to catch up with the ever-evolving source code base, its size and
An empirical analysis of flaky tests
TLDR
This work studies in detail a total of 201 commits that likely fix flaky tests in 51 open-source projects and identifies approaches that could manifest flaky behavior, and describes common strategies that developers use to fixFlaky tests.
Regression test selection for Java software
TLDR
A safe regression-test-selection technique that, based on the use of a suitable representation, handles the features of the Java language and also handles incomplete programs.
...
1
2
3
...