The Scientific Method in Practice: Reproducibility in the Computational Sciences

@inproceedings{Stodden2010TheSM,
  title={The Scientific Method in Practice: Reproducibility in the Computational Sciences},
  author={V. Stodden},
  year={2010}
}
Since the 1660’s the scientific method has included reproducibility as a mainstay in its effort to root error from scientific discovery. [...] Key Result It provides evidence in the debate about whether scientists’ research revealing behavior is wholly governed by considerations of personal impact or whether the reasoning behind the revealing decision involves larger scientific ideals, and secondly, this research describes the actual sharing behavior in the Machine Learning community.Expand
Research method in AI
Reproducibility of published computational research has seen increased interest the last twenty years. Regardless of academic field and the impact-factor of journals, studies of reproducibility ofExpand
R Markdown
Reproducibility is increasingly important to statistical research [16], but many details are often omitted from the published version of complex statistical analyses. A reader’s comprehension isExpand
A Model-Driven Engineering Framework for Computational Replicability and Reproducibility
TLDR
It is shown how the process has been used to successfully produce a Platform Independent Model from the essential components of a model developed in a Platform specific environment and the details of how this Platform Independent representation can be used to generate a second model in the original platform are confirmed. Expand
Performing reproductions to understand the state of reproducibility in current AI research
TLDR
This project attempts to provide a better understanding of the state of reproducibility in the field, and identifies issues limiting reproductions in AI from recent years. Expand
A Minimum Standard for Publishing Computational Results in the Weather and Climate Sciences
TLDR
A procedure for reporting computational results that was employed in a recent Journal of Climate paper is described, developed to be consistent with recommended computational best practices and to address the reproducibility crisis. Expand
METHODS TO ENSURE THE REPRODUCIBILITY OF BIOMEDICAL RESEARCH.
TLDR
In this commentary, the New England Journal of Medicine published a commentary calling scientists who repurpose data "research parasites" who misuse data generated by others to demonstrate alternative hypotheses. Expand
Towards standard practices for sharing computer code and programs in neuroscience
TLDR
This article lists practical suggestions to maximise the reproducibility of authors' work, and how best to maximize the chances that other scientists can reproduce the authors' computations or reuse their methods on their data. Expand
Computational reproducibility in geoscientific papers: Insights from a series of studies with geoscientists and a reproduction study
TLDR
Advice is proposed for authors to overcome the issues around reproducibility in the computational geosciences using the R code. Expand
Toward Reproducibility in Online Social Network Research
TLDR
It is found that sharing data are associated with increased citation count, while sharing method and code does not appear to be, and the architecture which supports the conduct of reproducible OSN research is introduced. Expand
Assessing the impact of introductory programming workshops on the computational reproducibility of biomedical workflows
TLDR
While none of the participants completely changed their workflows, many of them did incorporate new practices, tools, or methods that helped make their work more reproducible and transparent to other researchers, indicating that programming workshops now offered by libraries and other organizations contribute to computational reproducibility training for researchers. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 58 REFERENCES
15 Years of Reproducible Research in Computational Harmonic Analysis
TLDR
The approach to reproducible computational research is reviewed, how the approach has spread over time, and some of the arguments for and against working reproducibly are discussed. Expand
Enabling Reproducible Research: Open Licensing for Scientific Innovation
TLDR
The Reproducible Research Standard is proposed both to ensure attribution and facilitate the sharing of scientific works to encourage reproducible scientific investigation, facilitate greater collaboration, and promote engagement of the larger community in scientific learning and discovery. Expand
Laboratory life. The social construction of scientific facts
supported by facts, he believed in the good of the inductive method in all walks of life. He put this to work in studies such as the influences of environment on parameters of behaviour of mice andExpand
How Open Source Software Works: 'Free' User-to-User Assistance?
TLDR
It is found that the Apache field support system functions effectively and that, when the help system is partitioned into its component tasks, 98% of the effort expended by information providers in fact returns direct learning benefits to those providers. Expand
How Much Science Can You Have at Your Fingertips?
  • I. Good
  • Computer Science
  • IBM J. Res. Dev.
  • 1958
TLDR
Numerous suggestions are presented for helping men to learn and retrieve scientific information, with or without artificial aids, and a warning that some of the world's greatest scientists have done their best work when they were given plenty of time to think their own thoughts. Expand
The Structure of Scientific Revolutions
A good book may have the power to change the way we see the world, but a great book actually becomes part of our daily consciousness, pervading our thinking to the point that we take it for granted,Expand
Do Open-Access Articles Have a Greater Research Impact?
Although many authors believe that their work has a greater research impact if it is freely available, studies to demonstrate that impact are few. This study looks at articles in four disciplines atExpand
Breakdown Point of Model Selection When the Number of Variables Exceeds the Number of Observations
  • D. Donoho, V. Stodden
  • Mathematics, Computer Science
  • The 2006 IEEE International Joint Conference on Neural Network Proceedings
  • 2006
TLDR
This work points out that when p > n, there is a breakdown point for standard model selection schemes, such that model selection only works well below a certain critical complexity level depending on n/p, and applies this notion to some model selection algorithms (Forward Stepwise, LASSO, LARS) in the case where pGtn. Expand
Science of Science and Reflexivity
Over the last four decades, the French sociologist Pierre Bourdieu produced one of the most imaginative and subtle bodies of social theory of the postwar era. When he died in 2002, he was consideredExpand
Toward a new economics of science
Abstract Science policy issues have recently joined technology issues in being acknowledged to have strategic importance for national ‘competitiveness’ and ‘economic security’. The economicsExpand
...
1
2
3
4
5
...