Does preregistration improve the credibility of research findings?

  title={Does preregistration improve the credibility of research findings?},
  author={Mark Rubin},
  journal={arXiv: Other Statistics},
  • Mark Rubin
  • Published 20 October 2020
  • Psychology, Mathematics
  • arXiv: Other Statistics
Preregistration entails researchers registering their planned research hypotheses, methods, and analyses in a time-stamped document before they undertake their data collection and analyses. This document is then made available with the published research report to allow readers to identify discrepancies between what the researchers originally planned to do and what they actually ended up doing. This historical transparency is supposed to facilitate judgments about the credibility of the… 

Tables from this paper

Making ERP research more transparent: Guidelines for preregistration.
An overview of the problems associated with undisclosed analytic flexibility is presented, why and how EEG researchers would benefit from adopting preregistration are discussed, and guidelines and examples on how to preregister data preprocessing and analysis steps in typical ERP studies are provided.
Improving Evidence-Based Practice through Preregistration of Applied Research: Barriers and Recommendations.
The current review considers the potential of preregistration to improve both the accessibility and credibility of applied research towards more rigorous evidence-based practice.
When to adjust alpha during multiple testing: a consideration of disjunction, conjunction, and individual testing
Scientists often adjust their significance threshold (alpha level) during null hypothesis significance testing in order to take into account multiple testing and multiple comparisons. This alpha
There’s No Need to Lower the Significance Threshold when Conducting Single Tests of Multiple Individual Hypotheses
In summary, it is only necessary to lower the significance threshold when undertaking multiple tests of a single joint null hypothesis using an union-intersection approach. It is not necessary to
Responsible Science Matters


Does preregistration improve the credibility of research findings?
It is concluded that preregistration's historical transparency does not facilitate judgments about the credibility of research findings when researchers provide contemporary transparency in the form of clear rationales for current hypotheses and analytical approaches.
The effect of preregistration on trust in empirical research findings: results of a registered report
An empirical assessment of the extent to which preregistration increases the trust of 209 active academics in the reported outcomes, and how familiarity with another researcher influences that trust is presented.
The preregistration revolution
Widespread adoption of preregistration will increase distinctiveness between hypothesis generation and hypothesis testing and will improve the credibility of research findings.
The Value of Preregistration for Psychological Science: A Conceptual Analysis
For over two centuries researchers have been criticized for using research practices that makes it easier to present data in line with what they wish to be true. With the rise of the internet it has
An Agenda for Purely Confirmatory Research
This article proposes that researchers preregister their studies and indicate in advance the analyses they intend to conduct, and proposes that only these analyses deserve the label “confirmatory,” and only for these analyses are the common statistical tests valid.
Pre-analysis Plans Have Limited Upside, Especially Where Replications Are Feasible
It is argued that enthusiasm for pre-analysis plans should be tempered for several reasons, and practical proposals for how to increase the incentives for researchers to carry out replications are offered.
When Does HARKing Hurt? Identifying When Different Types of Undisclosed Post Hoc Hypothesizing Harm Scientific Progress
Hypothesizing after the results are known, or HARKing, occurs when researchers check their research results and then add or remove hypotheses on the basis of those results without acknowledging this
Results Blind Science Publishing
  • J. Locascio
  • Psychology, Medicine
    Basic and applied social psychology
  • 2017
A policy of “results blind evaluation” of manuscripts submitted to journals is recommended, in which results reported in manuscripts are given no weight in the decision as to the suitability of the manuscript for publication.
Detecting and avoiding likely false‐positive findings – a practical guide
It is argued that a culture of ‘you can publish if your study is rigorous’ creates a systematic bias against the null hypothesis which renders meta‐analyses questionable and may even lead to a situation where hypotheses become difficult to falsify.
Standard Operating Procedures: A Safety Net for Pre-Analysis Plans
ABSTRACT Across the social sciences, growing concerns about research transparency have led to calls for pre-analysis plans (PAPs) that specify in advance how researchers intend to analyze the data