An Analysis of 35+ Million Jobs of Travis CI

  title={An Analysis of 35+ Million Jobs of Travis CI},
  author={Monperrus Martin and Thomas Durieux and Rui Abreu and Tegawend{\'e} F. Bissyand{\'e} and Luis Cruz},
  journal={2019 IEEE International Conference on Software Maintenance and Evolution (ICSME)},
Travis CI handles automatically thousands of builds every day to, amongst other things, provide valuable feedback to thousands of open-source developers. In this paper, we investigate Travis CI to firstly understand who is using it, and when they start to use it. Secondly, we investigate how the developers use Travis CI and finally, how frequently the developers change the Travis CI configurations. We observed during our analysis that the main users of Travis CI are corporate users such as… 

Tables from this paper

Empirical Study of Restarted and Flaky Builds on Travis CI
This study analyzes restarted builds, flaky builds, and their impact on the development workflow to suggest that developers wait for CI results, interrupting their workflow to address the issue.
Lessons from Eight Years of Operational Datafrom a Continuous Integration Service
Continuous Integration (CI) is a popular practice that enables the rapid pace of modern software development. Cloud-based CI services have made CI ubiquitous by relieving software teams of the hassle
Software Batch Testing to Reduce Build Test Executions
The goal of this work is to improve the efficiency of the batch testing and recommends that all CI pipelines use a batch size of at least four, the greatest reduction is BatchStop4 at 50%.
PerfCI: A Toolchain for Automated Performance Testing during Continuous Integration of Python Projects
A toolchain - PerfCI - to pave the way for developers to easily set up and carry out automated performance testing under CI and demonstrate the feasibility of the toolchain by using it on a web service running at the Compact Muon Solenoid experiment at the world's largest particle physics laboratory - CERN.
R-testbench: a journey in open source programming for remote instrumentation with Python
R-testbench is an open source Python library that offers highlevel programming capabilities for instrument remote control and test bench automation, and relies on the Virtual Instrumentation Software Architecture (VISA) standard, and enables high-level features, including automatic instrument recognition, thanks to the object-oriented paradigm.
FlexiRepair: Transparent Program Repair with Generic Patches
This paper presents the design and implementation of a repair framework, FLEXIREPAIR, that explores generic patches as the core concept and shows how concretely generic patches can be inferred and applied in a pipeline of Automated Program Repair (APR).
A Software-Repair Robot Based on Continual Learning
R-Hero, a novel system for learning how to fix bugs based on continual training, is discussed, which aims to reduce the time and money needed to correct software bugs.
Robbery on DevOps: Understanding and Mitigating Illicit Cryptomining on Continuous Integration Service Platforms
A systematic study on real-world illicit cryptomining on public CI platforms, using a novel technique, called Cijitter, to strategically inject delays to the execution of a CI workflow to disproportionally penalize the mining jobs that need to work on a series of tasks under time constraints.
Lessons from Eight Years of Operational Data from a Continuous Integration Service: An Exploratory Case Study of CircleCI
Continuous Integration (CI) is a popular practice that enables the rapid pace of modern software development. Cloud-based CI services have made CI ubiquitous by relieving software teams of the hassle
Fixing Continuous Integration Tests From Within the IDE With Contextual Information
This work investigates how presenting different kinds of contextual information about CI builds in the Integrated Development Environment (IDE) impacts the time developers take to fix a broken build and shows that TESTAXIS helps developers fix failing tests 13.4% to 48.6% faster.


TravisTorrent: Synthesizing Travis CI and GitHub for Full-Stack Research on Continuous Integration
In order to lay the groundwork for data-driven research on CI, TravisTorrent is built, a freely available data set based on Travis CI and GitHub that provides easy access to hundreds of thousands of analyzed builds from more than 1,000 projects.
Oops, My Tests Broke the Build: An Explorative Analysis of Travis CI with GitHub
It is found that testing is the single most important reason why builds fail on Travis CI, and the programming language has a strong influence on both the number of executed tests, their run time, and proneness to fail.
Usage, costs, and benefits of continuous integration in open-source projects
Evidence is shown that supports the claim that CI helps projects release more often, that CI is widely adopted by the most popular projects, as well as finding that the overall percentage of projects using CI continues to grow, making it important and timely to focus more research on CI.
I'm Leaving You, Travis: A Continuous Integration Breakup Story
This paper quantifies the effects that open-source community factors and project technical factors have on the rate of Travis abandonment, and finds that increased build complexity reduces the chances of abandonment, that larger projects abandon at higher rates, and that a project's dominant language has significant but varying effects.
An Empirical Analysis of Build Failures in the Continuous Integration Workflows of Java-Based Open-Source Software
An analysis of build failures in CI environments, using data from 14 open-source Java projects, indicates that process metrics have a significant impact on the build outcome in 8 of the 14 projects on average, but the strongest influencing factor across all projects is overall stability in the recent build history.
The impact of continuous integration on other software development practices: A large-scale empirical study
The adaptation and evolution of code writing and submission, issue and pull request closing, and testing practices as Travis CI is adopted by hundreds of established projects on GitHub are studied to suggest a more nuanced picture of how GitHub teams are adapting to, and benefiting from, continuous integration technology.
Continuous Integration
Go a new server for continuous integration and delivery, which supports parallel, deployment pipelines with multiple projects, a snazzy looking dashboard, and support for automated deployments.
Blockchain-Based Software Engineering
  • M. Beller, J. Hejderup
  • Computer Science
    2019 IEEE/ACM 41st International Conference on Software Engineering: New Ideas and Emerging Results (ICSE-NIER)
  • 2019
This paper defines how Blockchain technology could solve two core SE problems: Continuous Integration Services such as Travis CI and Package Managers such as apt-get and democratize and professionalize Software Engineering infrastructure that currently relies on free work done by few volunteers.
Continuous integration Blockchain - based software engineering , ” in Proceedings of the 41 th International Conference on Software Engineering : New Ideas and Emerging Results
  • Proceedings of the 14 th International Conference on Mining Software Repositories