The Value of Direct Replication

@article{Simons2014TheVO,
  title={The Value of Direct Replication},
  author={Daniel J. Simons},
  journal={Perspectives on Psychological Science},
  year={2014},
  volume={9},
  pages={76 - 80}
}
  • D. Simons
  • Published 1 January 2014
  • Psychology, Medicine
  • Perspectives on Psychological Science
Reproducibility is the cornerstone of science. If an effect is reliable, any competent researcher should be able to obtain it when using the same procedures with adequate statistical power. Two of the articles in this special section question the value of direct replication by other laboratories. In this commentary, I discuss the problematic implications of some of their assumptions and argue that direct replication by multiple laboratories is the only way to verify the reliability of an effect… 

Topics from this paper

What Is a Replication?
  • E. Machery
  • Computer Science
    Philosophy of Science
  • 2020
TLDR
It is argued that a replication is an experiment that resamples the experimental components of an original experiment that are treated as random factors and that the function of replications is, narrowly, to assess the reliability of the replicated experiments.
On Replication in Communication Science
TLDR
The importance of replications for communication science is outlined and a framework for this special issue on replications is provided, calling for communication scholars to consider future projects and structural changes that would incentivize future replication studies.
Changing the logic of replication: A case from infant studies.
TLDR
It is suggested that, in order to accumulate and generate new knowledge, the dichotomous view of replication as confirmatory/disconfirmatory can be replaced by an approach that emphasizes the estimation of effect sizes via meta-analysis.
Contextual sensitivity in scientific reproducibility
TLDR
It is found that the extent to which the research topic was likely to be contextually sensitive was associated with replication success, and this relationship remained a significant predictor of replication success even after adjusting for characteristics of the original and replication studies that previously had been associated with replicate success.
There's more than one way to conduct a replication study: Beyond statistical significance.
TLDR
A variety of additional "replication goals" are developed that will allow researchers to develop a more nuanced understanding of replication that can be flexible enough to answer the various questions that researchers might seek to understand.
A Causal Replication Framework for Designing and Assessing Replication Efforts
TLDR
This paper presents a meta-answers to the question of what constitutes a replication and how results from a replicated experiment can be compared to those obtained without a replication.
Understanding Replication: Trust But Verify
Replication as a pillar of science is described in the context of the replication crisis that first struck psychology but spread quickly to other science-based fields. Empirical evidence suggests
The Limits of Direct Replications and the Virtues of Stimulus Sampling
While direct replications such as the ‘‘Many Labs’’ project are extremely valuable in testing the reliability of published findings across laboratories, they reflect the common reliance in psychology
Statistical methods for evidence synthesis
TLDR
This dissertation considers three realms in which synthesis of modules of evidence can occur: when meta-analyzing multiple studies; when subjecting a single study to independent replications; and when testing related hypotheses within a study.
A Replication by Any Other Name
Replication research is essential to scientific knowledge. Reviews of replication studies often electronically search for replicat* as a textword, which does not identify studies that replicate
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 23 REFERENCES
The Replication Recipe: What Makes for a Convincing Replication?
Psychological scientists have recently started to reconsider the importance of close replications in building a cumulative knowledge base; however, there is no consensus about what constitutes a
Is the Replicability Crisis Overblown? Three Arguments Examined
  • H. Pashler, C. Harris
  • Sociology, Medicine
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2012
TLDR
It is argued that there are no plausible concrete scenarios to back up such forecasts and that what is needed is not patience, but rather systematic reforms in scientific practice.
Recommendations for Increasing Replicability in Psychology †
Replicability of findings is at the heart of any empirical science. The aim of this article is to move the current replicability debate in psychology towards concrete recommendations for improvement.
The Alleged Crisis and the Illusion of Exact Replication
  • W. Stroebe, F. Strack
  • Medicine, Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2014
TLDR
It is proposed that for meaningful replications, attempts at reinstating the original circumstances are not sufficient and replicators must ascertain that conditions are realized that reflect the theoretical variable(s) manipulated (and/or measured) in the original study.
Priming, Replication, and the Hardest Science
  • Joseph Cesario
  • Psychology, Medicine
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2014
TLDR
It is argued that expectations of priming effects to be highly sensitive to variations in experimental features and subject populations are misguided and inconsistent with evolutionary understandings of the brain as a computational organ.
False-Positive Psychology
TLDR
It is shown that despite empirical psychologists’ nominal endorsement of a low rate of false-positive findings, flexibility in data collection, analysis, and reporting dramatically increases actual false- positive rates, and a simple, low-cost, and straightforwardly effective disclosure-based solution is suggested.
Measuring the Prevalence of Questionable Research Practices with Incentives for Truth-Telling
Cases of clear scientific misconduct have received significant media attention recently, but less flagrantly questionable research practices may be more prevalent and, ultimately, more damaging to
Measuring the Prevalence of Questionable Research Practices With Incentives for Truth Telling
TLDR
It is found that the percentage of respondents who have engaged in questionable practices was surprisingly high, which suggests that some questionable practices may constitute the prevailing research norm.
BELIEF IN THE LAW OF SMALL NUMBERS
“Suppose you have run an experiment on 20 subjects, and have obtained a significant result which confirms your theory ( z = 2.23, p If you feel that the probability is somewhere around .85, you may
“Positive” Results Increase Down the Hierarchy of the Sciences
TLDR
These results support the scientific status of the social sciences against claims that they are completely subjective, by showing that, when they adopt a scientific approach to discovery, they differ from the natural sciences only by a matter of degree.
...
1
2
3
...