Does preregistration improve the credibility of research findings?

@article{Rubin2020DoesPI,
  title={Does preregistration improve the credibility of research findings?},
  author={Mark Rubin},
  journal={arXiv: Other Statistics},
  year={2020}
}
  • Mark Rubin
  • Published 20 October 2020
  • Psychology, Mathematics
  • arXiv: Other Statistics
Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the… Expand
View PDF on arXiv
5 Citations
Highly Influential Citations
1
Background Citations
2

Tables from this paper

5 Citations

Citation Type

Citation Type

Making ERP research more transparent: Guidelines for preregistration.
TLDR
An overview of the problems associated with undisclosed analytic flexibility is presented, why and how EEG researchers would benefit from adopting preregistration are discussed, and guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies are provided. Expand
Improving Evidence-Based Practice through Preregistration of Applied Research: Barriers and Recommendations.
TLDR
The current review considers the potential of preregistration to improve both the accessibility and credibility of applied research towards more rigorous evidence-based practice. Expand
When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing
Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha… Expand
There’s No Need to Lower the Significance Threshold when Conducting Single Tests of Multiple Individual Hypotheses
In summary, it is only necessary to lower the significance threshold when undertaking multiple tests of a single joint null hypothesis using an union-intersection approach. It is not necessary to… Expand
Responsible Science Matters

References

SHOWING 1-10 OF 111 REFERENCES
Does preregistration improve the credibility of research findings?
TLDR
It is concluded that preregistration's historical transparency does not facilitate judgments about the credibility of research findings when researchers provide contemporary transparency in the form of clear rationales for current hypotheses and analytical approaches. Expand
The effect of preregistration on trust in empirical research findings: results of a registered report
TLDR
An empirical assessment of the extent to which preregistration increases the trust of 209 active academics in the reported outcomes, and how familiarity with another researcher influences that trust is presented. Expand
The preregistration revolution
TLDR
Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings. Expand
An Agenda for Purely Confirmatory Research
TLDR
This article proposes that researchers preregister their studies and indicate in advance the analyses they intend to conduct, and proposes that only these analyses deserve the label “confirmatory,” and only for these analyses are the common statistical tests valid. Expand
Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible
TLDR
It is argued that enthusiasm for pre-analysis plans should be tempered for several reasons, and practical proposals for how to increase the incentives for researchers to carry out replications are offered. Expand
When Does HARKing Hurt? Identifying When Different Types of Undisclosed Post Hoc Hypothesizing Harm Scientific Progress
Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this… Expand
Results Blind Science Publishing
  • J. Locascio
  • Psychology, Medicine
  • Basic and applied social psychology
  • 2017
TLDR
A policy of “results blind evaluation” of manuscripts submitted to journals is recommended, in which results reported in manuscripts are given no weight in the decision as to the suitability of the manuscript for publication. Expand
Detecting and avoiding likely false‐positive findings – a practical guide
TLDR
It is argued that a culture of ‘you can publish if your study is rigorous’ creates a systematic bias against the null hypothesis which renders meta‐analyses questionable and may even lead to a situation where hypotheses become difficult to falsify. Expand
Standard Operating Procedures: A Safety Net for Pre-Analysis Plans
ABSTRACT Across the social sciences, growing concerns about research transparency have led to calls for pre-analysis plans (PAPs) that specify in advance how researchers intend to analyze the data… Expand
An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Selectively publishing results that support the tested hypotheses (“positive” results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been… Expand
...
1
2
3
4
5
...