Build scripts with perfect dependencies

@article{Spall2020BuildSW,
  title={Build scripts with perfect dependencies},
  author={Sarah Spall and Neil Mitchell and Sam Tobin-Hochstadt},
  journal={Proceedings of the ACM on Programming Languages},
  year={2020},
  volume={4},
  pages={1 - 28}
}
Build scripts for most build systems describe the actions to run, and the dependencies between those actions - but often build scripts get those dependencies wrong. Most build scripts have both too few dependencies (leading to incorrect build outputs) and too many dependencies (leading to excessive rebuilds and reduced parallelism). Any programmer who has wondered why a small change led to excess compilation, or who resorted to a clean step, has suffered the ill effects of incorrect dependency… 
View on ACM
dl.acm.org
5 Citations
Background Citations
5
Methods Citations
1

5 Citations

LaForge: Always-Correct and Fast Incremental Builds from Simple Specifications

This work introduces LAFORGE, a novel build tool that eliminates the need to specify dependencies or incremental build steps, and shows that build specifications can be both simple and fast.

Riker: Always-Correct and Fast Incremental Builds from Simple Specifications

R IKER is a build system that automatically discovers and runs incremental builds from simple specifications, which makes it easy to reproduce the experiments described in the paper, which evaluate R IKER’s performance and effectiveness as a build tool.

Accelerating Build Dependency Error Detection via Virtual Build

The tool VirtualBuild is implemented and it is shown that it detects all the dependency errors found by existing tools at a low cost and accelerates the build process by 8.74 times, and improves the efficiency of error detection by 6.13 times on average.

Forward build systems, formally

This paper defines what it means for a forward build system to be correct, and shows that other forward build systems, such as Fabricate and Memoize, are also correct.

Demystifying Dependency Bugs in Deep Learning Stack

Findings from thisstudyshed lightontheimplicationsondependencymanagement,e.g.,constructingdependencyknowledgegraphfortheentireDLstack,recommendingdependencies,detecting, localizing andfixing de-pendencybugs, and upgrading andmigratingdependencies.

References

SHOWING 1-10 OF 50 REFERENCES

Shake before building: replacing make with haskell

This work shows how to eliminate the dependency restriction in build tools, allowing additional dependencies to be specified while building, and has implemented the ideas in the Haskell library Shake, and used Shake to write a complex build system which compiles millions of lines of code.

Scalable Incremental Building with Dynamic Task Dependencies

A new incremental build algorithm is presented that allows build engineers to use a full-fledged programming language with explicit task invocation, value and file inspection facilities, and conditional and iterative language constructs and is independent of the size of the software project being built.

A sound and optimal incremental build system with dynamic dependencies

This work presents a build system called pluto that supports the definition of reusable, parameterized, interconnected builders and generalizes the traditional notion of time stamps to allow builders to declare their actual requirements on a file's content.

Non-recursive make considered harmful: build systems at scale

Functional programming is used to design abstractions for build systems, and implemented on top of the Shake library, which allows them to describe build rules and dependencies, which is more scalable, faster, and spectacularly more maintainable than its Make-based predecessor.

An empirical study of unspecified dependencies in make-based build systems

A case study on unspecified dependencies in the make-based build systems of the glib, openldap, linux and qt open source projects is presented and it is found that unspecified dependencies are common.

Non-recursive Make Considered Harmful

Functional programming is used to design abstractions for build systems, and implemented on top of the Shake library, which allows them to describe build rules and dependencies, which is more scalable, faster, and spectacularly more maintainable than its Make-based predecessor.

PIE: A Domain-Specific Language for Interactive Software Development Pipelines

PIE is developed, a Domain-Specific Language (DSL), API, and runtime for developing interactive software development pipelines, that enables direct and concise expression of pipelines without boilerplate, reducing the development and maintenance effort of pipelines.

Recursive Make Considered Harmful

This paper explores a number of problems regarding the use of recursive make, and shows that they are all symptoms of the same problem, and the problem stems from the artificial partitioning of the build into separate subsets.

Make — a program for maintaining computer programs

  • S. Feldman
  • Computer Science, Education
    Softw. Pract. Exp.
  • 1979
A program that can keep track of the relationships between parts of a program, and issue the commands needed to make the parts consistent after changes are made is described.

Safe futures for Java

The definition and implementation of safe futures for Java are explored and it is indicated that for programs with modest mutation rates on shared data, applications can use futures to profitably exploit parallelism, without sacrificing safety.