An empirical analysis of flaky tests

@article{Luo2014AnEA,
  title={An empirical analysis of flaky tests},
  author={Q. Luo and F. Hariri and Lamyaa Eloussi and D. Marinov},
  journal={Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering},
  year={2014}
}
  • Q. Luo, F. Hariri, +1 author D. Marinov
  • Published 2014
  • Computer Science, Engineering
  • Proceedings of the 22nd ACM SIGSOFT International Symposium on Foundations of Software Engineering
Regression testing is a crucial part of software development. [...] Key Method We study in detail a total of 201 commits that likely fix flaky tests in 51 open-source projects. We classify the most common root causes of flaky tests, identify approaches that could manifest flaky behavior, and describe common strategies that developers use to fix flaky tests. We believe that our insights and implications can help guide future research on the important topic of (avoiding) flaky tests.Expand
205 Citations
Does Refactoring of Test Smells Induce Fixing Flaky Tests?
  • F. Palomba, A. Zaidman
  • Engineering, Computer Science
  • 2017 IEEE International Conference on Software Maintenance and Evolution (ICSME)
  • 2017
  • 73
A Study on the Lifecycle of Flaky Tests
  • 10
  • Highly Influenced
  • PDF
Understanding Reproducibility and Characteristics of Flaky Tests Through Test Reruns in Java Projects
  • 4
  • PDF
Determining flaky tests from test failures
  • 5
  • PDF
De-Flake Your Tests : Automatically Locating Root Causes of Flaky Tests in Code At Google
  • 1
  • Highly Influenced
Detecting and Fixing Flaky Tests
  • PDF
The smell of fear: on the relation between test smells and flaky tests
  • 31
  • Highly Influenced
  • PDF
Flaky Tests: Problems, Solutions, and Challenges
  • PDF
DeFlaker: Automatically Detecting Flaky Tests
  • 69
  • PDF
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 14 REFERENCES
Empirically revisiting the test independence assumption
  • 103
  • Highly Influential
  • PDF
Finding bugs by isolating unit tests
  • 42
  • Highly Influential
  • PDF
Automated testing of GUI applications: Models, tools, and controlling flakiness
  • A. Memon, M. Cohen
  • Computer Science
  • 2013 35th International Conference on Software Engineering (ICSE)
  • 2013
  • 42
  • Highly Influential
  • PDF
Unit test virtualization with VMVM
  • 64
  • Highly Influential
  • PDF
Killing the Gatekeeper: Introducing a Continuous Integration System
  • 29
  • Highly Influential
Continuous integration at Google scale
  • Continuous integration at Google scale
  • 2013
Testing at the speed and scale of Google
  • Testing at the speed and scale of Google
  • 2011
Swing Extreme Testing
  • Swing Extreme Testing
  • 2008
Android FlakyTest annotation
  • Android FlakyTest annotation
Eradicating non-determinism in tests
  • Eradicating non-determinism in tests
...
1
2
...