Trade-offs in continuous integration: assurance, security, and flexibility

@article{Hilton2017TradeoffsIC,
  title={Trade-offs in continuous integration: assurance, security, and flexibility},
  author={Michael C Hilton and Nicholas Nelson and Timothy Tunnell and Darko Marinov and Danny Dig},
  journal={Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering},
  year={2017}
}
Continuous integration (CI) systems automate the compilation, building, and testing of software. [] Key Method We conduct semi-structured interviews with developers from different industries and development scales. We triangulate our findings by running two surveys. We find that developers face trade-offs between speed and certainty (Assurance), between better access and information security (Security), and between more configuration options and greater ease of use (Flexi- bility). We present implications of…

Figures and Tables from this paper

Design and Implementation of SFCI: A Tool for Security Focused Continuous Integration
TLDR
This paper introduces the design and implementation of a tool for security-focused continuous integration, primarily focused on integrating third party security testing programs and generating reports on classes of vulnerabilities found in a software project.
Work practices and challenges in continuous integration: A survey with Travis CI users
TLDR
A user survey with 158 CI users finds that developers are not sure whether a job failure represents a failure or not, and inadequate testing is the most common technical reason related to build breakage.
Understanding and Improving Continuous Integration and Delivery Practice using Data from the Wild
TLDR
This talk will outline recent work aimed at characterizing theCI/CD process from different perspectives, and provide an overview of various kinds of anti-patterns related to the CI/CD adoption.
Characterizing The Influence of Continuous Integration
TLDR
The findings indicate that only adoption of CI might not be enough to the improve software development process, and recommends industry practitioners to adopt the best practices of CI to reap the benefits of CI tools for example, making frequent commits.
The current practices of changing secure software: an empirical study
TLDR
The study suggests that developers of secure software need techniques that support effective security assurance of modified software, including code review, code analysis, testing, and keywords search.
An empirical characterization of bad practices in continuous integration
TLDR
This paper empirically investigates what are the bad practices experienced by developers applying Continuous Integration, and compiled a catalog of 79 CI bad smells belonging to 7 categories related to different dimensions of a CI pipeline management and process.
A Survey on Developer-Centred Security
TLDR
This work provides a systematised overview of the relatively new field of Developer-Centred Security which aims to understand the context in which developers produce security-relevant code as well as provide tools and processes that that better support both developers and secure code production.
Characterizing the influence of continuous integration: empirical results from 250+ open source and proprietary projects
TLDR
The findings indicate that only adoption of CI might not be enough to the improve software development process, and recommend industry practitioners to adopt the best practices of CI to reap the benefits of CI tools for example, making frequent commits.
Reuse (or Lack Thereof) in Travis CI Specifications: An Empirical Study of CI Phases and Commands
TLDR
An empirical analysis of the use of different phases and commands in a curated sample of 913 CI specifications for Java-based projects that use Travis CI suggests that the usage of phases and Commands in Travis CI specifications are broad and diverse.
A framework for detecting and preventing security vulnerabilities in continuous integration/continuous delivery pipelines
TLDR
This paper aims at delivering a framework for detecting and preventing security vulnerabilities in Continuous Integration/Continuous Delivery pipelines in the context of a large consultancy company which provides Continuous Integration and Continuous Delivery environments as a service to customers and internal development teams.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 55 REFERENCES
Usage, costs, and benefits of continuous integration in open-source projects
TLDR
Evidence is shown that supports the claim that CI helps projects release more often, that CI is widely adopted by the most popular projects, as well as finding that the overall percentage of projects using CI continues to grow, making it important and timely to focus more research on CI.
Why Do Automated Builds Break? An Empirical Study
TLDR
The main factors impacting build breakage are quantitatively investigated and found that build failures correlate with the number of simultaneous contributors on branches, the type of work items performed on a branch, and the roles played by the stakeholders of the builds.
Expectations, outcomes, and challenges of modern code review
TLDR
This study reveals that while finding defects remains the main motivation for review, reviews are less about defects than expected and instead provide additional benefits such as knowledge transfer, increased team awareness, and creation of alternative solutions to problems.
How do software engineers understand code changes?: an exploratory study in industry
TLDR
The role of understanding code changes during software-development process is investigated, engineers' information needs for understanding changes and their requirements for the corresponding tool support are explored, and a number of insufficiencies in the current practice are reinforced.
Towards quality gates in continuous delivery and deployment
TLDR
This work provides a model for this trade-off of release “confidence” and “velocity” that led to the formulation of 4 categories (cautious, balanced, problematic, madness), in which companies can be classified in.
Quality and productivity outcomes relating to continuous integration in GitHub
TLDR
The main finding is that continuous integration improves the productivity of project teams, who can integrate more outside contributions, without an observable diminishment in code quality.
Techniques for improving regression testing in continuous integration development environments
TLDR
This work presents algorithms that make continuous integration processes more cost-effective, involving algorithms that are relatively inexpensive and do not rely on code coverage information -- two requirements for conducting testing cost- effective in this context.
Security practices in DevOps
TLDR
Bringing security principles within the DevOps process can help the organization in achieving better quality of software by integrating security checks into the phases of development, testing, and deployment.
Challenges When Adopting Continuous Integration: A Case Study
TLDR
A case study is presented in which the challenges of continuous integration are assessed based on interviews at a major Swedish telecommunication services and equipment provider and the study found 23 adoption challenges that organisations may face when adopting the continuous integration process.
Software history under the lens: A study on why and how developers examine it
TLDR
It is found that history does not begin with the latest commit but with uncommitted changes, and a novel unified model for reasoning about software history, 3-LENS HISTORY is proposed.
...
1
2
3
4
5
...