Extreme mutation testing in practice: An industrial case study

  title={Extreme mutation testing in practice: An industrial case study},
  author={Maik Betka and Stefan Wagner},
  journal={2021 IEEE/ACM International Conference on Automation of Software Test (AST)},
  • Maik Betka, S. Wagner
  • Published 15 March 2021
  • Computer Science
  • 2021 IEEE/ACM International Conference on Automation of Software Test (AST)
Mutation testing is used to evaluate the effectiveness of test suites. In recent years, a promising variation called extreme mutation testing emerged that is computationally less expensive. It identifies methods where their functionality can be entirely removed, and the test suite would not notice it, despite having coverage. These methods are called pseudo-tested. In this paper, we compare the execution and analysis times for traditional and extreme mutation testing and discuss what they mean… 

Tables from this paper


Will My Tests Tell Me If I Break This Code?
An extreme mutation testing approach is applied to analyze the tests of open-source projects written in Java and shows that the ratio of pseudo-tested methods is acceptable for unit tests but not for system tests (that execute large portions of the whole system).
Evaluation and improvement of automated software test suites
This dissertation proposed, implemented and evaluated a light-weight mutation approach to identify pseudo-tested methods; that is, methods that are less resource-intensive than currently used mutation testing approaches.
Chapter Six - Mutation Testing Advances: An Analysis and Survey
This chapter presents a survey of recent advances, over the past decade, related to the fundamental problems of mutation testing and sets out the challenges and open problems for the future development of the method.
Demo: PIT a Practical Mutation Testing Tool for Java
Mutation analysis introduces program defects with the intend of verifying whether candidate tests are able to trigger anomalous behaviour. In case the tests can distinguish the defective behaviour
An Analysis and Survey of the Development of Mutation Testing
  • Yue Jia, M. Harman
  • Computer Science
    IEEE Transactions on Software Engineering
  • 2011
These analyses provide evidence that Mutation Testing techniques and tools are reaching a state of maturity and applicability, while the topic of Mutation testing itself is the subject of increasing interest.
A comprehensive study of pseudo-tested methods
In-depth characterization of pseudo-tested methods leads to two key insights: pseudo- tested methods are significantly less tested than the other methods; yet, for most of them, the developers would not pay the testing price to fix this situation.
Descartes: A PITest Engine to Detect Pseudo-Tested Methods: Tool Demonstration
The demonstration compares Descarte with Gregor, the default mutation engine provided by PITest, in a set of real open source projects and provides some insights on the main features exposed by Descartes.
Mutation testing advances : an analysis and survey , ” in
  • Advances in Computers