Enhancing reproducibility for computational methods
@article{Stodden2016EnhancingRF, title={Enhancing reproducibility for computational methods}, author={Victoria Stodden and Marcia K. McNutt and David H. Bailey and Ewa Deelman and Yolanda Gil and Brooks Hanson and M. Heroux and John P. A. Ioannidis and Michela Taufer}, journal={Science}, year={2016}, volume={354}, pages={1240 - 1241} }
Data, code, and workflows should be available and cited Over the past two decades, computational methods have radically changed the ability of researchers from all areas of scholarship to process and analyze data and to simulate complex systems. But with these advances come challenges that are contributing to broader concerns over irreproducibility in the scholarly literature, among them the lack of transparency in disclosure of computational methods. Current reporting methods are often uneven…
254 Citations
Reproducibility in Research: Systems, Infrastructure, Culture
- Computer Science
- 2015
A high-level prototype open automated platform for scientific software development which effectively abstracts specific dependencies from the individual researcher and their workstation, allowing easy sharing and reproduction of results is proposed.
Publishing computational research - A review of infrastructures for reproducible and transparent scholarly communication
- Computer ScienceArXiv
- 2020
A review of software solutions designed to solve the problem of publishing executable computational research results and properties related to preservation relevant for librarians dealing with long-term accessibility of research materials are included.
Computing Environments for Reproducibility: Capturing the "Whole Tale"
- Computer ScienceFuture Gener. Comput. Syst.
- 2019
An empirical analysis of journal policy effectiveness for computational reproducibility
- MedicineProceedings of the National Academy of Sciences
- 2018
This work evaluates the effectiveness of journal policy that requires the data and code necessary for reproducibility be made available postpublication by the authors upon request and finds it to be an improvement over no policy, but currently insufficient for reproducecibility.
Reproducible Research in Computational Chemistry of Materials
- Computer Science
- 2017
The different aspects of research reproducibility in computational chemistry and materials science are summarized and the current practices of the community (both researchers and publishers) in this respect are delineated.
A Practical Taxonomy of Reproducibility for Machine Learning Research
- Computer Science
- 2018
A taxonomy of reproducibility is discussed from the perspective of a practitioner who sets out to apply a new computational method within their own domain, those which provide the code, data, and full computational environment necessary to reproduce the results of the study.
A Practical Taxonomy of Reproducibility for Machine Learning Research
- Computer Science
- 2018
A taxonomy of reproducibility is discussed from the perspective of a practitioner who sets out to apply a new computational method within their own domain, those which provide the code, data, and full computational environment necessary to reproduce the results of the study.
Epistemic issues in computational reproducibility: software as the elephant in the room
- Computer ScienceEuropean Journal for Philosophy of Science
- 2021
It is argued that the epistemic issues at stake in actual practices of computational reproducedcibility are best unveiled by focusing on software as a pivotal concept, one that is surprisingly often overlooked in accounts of reproducibility issues.
Enabling the Verification of Computational Results: An Empirical Evaluation of Computational Reproducibility
- Computer ScienceProceedings of the First International Workshop on Practical Reproducible Evaluation of Computer Systems
- 2018
This article attempts to regenerate published claims from author-supplied information, including data, code, inputs, and other provided specifications, on a different computational system than that used by the original authors.
Successes and Struggles with Computational Reproducibility: Lessons from the Fragile Families Challenge
- BiologySocius: Sociological Research for a Dynamic World
- 2019
The authors describe their approach to enabling computational reproducibility for the 12 articles in this special issue of Socius about the Fragile Families Challenge, and draw on two tools commonly used by professional software engineers but not widely used by academic researchers: software containers and cloud computing.
References
SHOWING 1-10 OF 12 REFERENCES
Liberating field science samples and data
- Environmental ScienceScience
- 2016
Transparency and reproducibility enhance the integrity of research results for scientific and public uses and empower novel research applications and ways in which funders, publishers, scientific societies, and others are responding are discussed.
Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research
- Computer Science
- 2014
A formalized set of best practice recommendations for computational scientists wishing to disseminate reproducible research, facilitate innovation by enabling data and code re-use, and enable broader communication of the output of digital scientific research are presented.
Quantifying Reproducibility in Computational Biology: The Case of the Tuberculosis Drugome
- Computer SciencePloS one
- 2013
This paper attempts to quantify this difficulty of reproducibility by reproducing a previously published paper for different classes of users (ranging from users with little expertise to domain experts) and suggest ways in which the situation might be improved.
The Legal Framework for Reproducible Scientific Research: Licensing and Copyright
- Computer ScienceComputing in Science & Engineering
- 2009
The author proposes the reproducible research standard for scientific researchers to use for all components of their scholarship that should encourage reproducible scientific investigation through attribution, facilitate greater collaboration, and promote engagement of the larger community in scientific learning and discovery.
Statistical Analyses and Reproducible Research
- Computer Science
- 2007
This article describes a software framework for both authoring and distributing integrated, dynamic documents that contain text, code, data, and any auxiliary content needed to recreate the computations in data analyses, methodological descriptions, simulations, and so on.
Toward Reproducible Computational Research: An Empirical Analysis of Data and Code Policy Adoption by Journals
- Computer SciencePloS one
- 2013
A predictive model of open data and code policy adoption as a function of impact factor and publisher is built and it is found higher impact journals more likely to have Open Data and Code Policy Day and scientific societies more likely than commercial publishers to have openData and code policies.
Public Availability of Published Research Data in High-Impact Journals
- Medicine, Political SciencePloS one
- 2011
A substantial proportion of original research papers published in high-impact journals are either not subject to any data availability policies, or do not adhere to the data availability instructions in their respective journals.
Promoting an open research culture
- MedicineScience
- 2015
Author guidelines for journals could help to promote transparency, openness, and reproducibility Transparency, openness, and reproducibility are readily recognized as vital features of science (1,…
The Princeton companion to applied mathematics
- Mathematics
- 2015
This volume introduces readers to applied mathematics and its uses; explains key concepts; describes important equations, laws, and functions; looks at exciting areas of research; covers modeling and simulation; explores areas of application; and more.
Best Practices for Computational Science: Software Infrastructure and Environments for Reproducible and Extensible Research
- Computer Science
- 2013
Scholarly dissemination and communication standards are changing to reflect the increasingly computational nature of scholarly research, primarily to include the sharing of the data and code…