Corpus ID: 14351731

Repeatability and Benefaction in Computer Systems Research — A Study and a Modest Proposal

  title={Repeatability and Benefaction in Computer Systems Research — A Study and a Modest Proposal},
  author={Alex M. Warren},
  • Alex M. Warren
  • Published 2015
  • We describe a study into the extent to which Computer Systems researchers share their code and data and the extent to which such code builds. Starting with 601 papers from ACM conferences and journals, we examine 402 papers whose results were backed by code. For 32.3% of these papers we were able to obtain the code and build it within 30 minutes; for 48.3% of the papers we managed to build the code, but it may have required extra effort; for 54.0% of the papers either we managed to build the… CONTINUE READING
    51 Citations
    Tackling the Reproducibility Problem in Systems Research with Declarative Experiment Specifications
    • PDF
    A collaborative approach to computational reproducibility
    • 8
    • PDF
    Sustainable computational science : the
    • 1
    • PDF
    On the Techniques We Create, the Tools We Build, and Their Misalignments: A Study of KLEE
    • 14
    Sustainable computational science: the ReScience initiative
    • 53
    • PDF
    PRUNE: A preserving run environment for reproducible scientific computing
    • Peter Ivie, D. Thain
    • Computer Science
    • 2016 IEEE 12th International Conference on e-Science (e-Science)
    • 2016
    • 2
    • PDF


    Repeatability, reproducibility and rigor in systems research
    • J. Vitek, T. Kalibera
    • Computer Science
    • 2011 Proceedings of the Ninth ACM International Conference on Embedded Software (EMSOFT)
    • 2011
    • 52
    • PDF
    Measuring Reproducibility in Computer Systems Research
    • 62
    • PDF
    Run your research: on the effectiveness of lightweight mechanization
    • 92
    • PDF
    A Universal Identifier for Computational Results
    • 52
    • PDF
    Reproducible Research: A Bioinformatics Case Study
    • R. Gentleman
    • Computer Science, Medicine
    • Statistical applications in genetics and molecular biology
    • 2005
    • 139
    • PDF
    Rex: A toolset for reproducing software experiments
    • 7
    • PDF