• Corpus ID: 232240576

Screening $p$-Hackers: Dissemination Noise as Bait

@inproceedings{Echenique2021ScreeningD,
  title={Screening \$p\$-Hackers: Dissemination Noise as Bait},
  author={Federico Echenique and Kevin He},
  year={2021}
}
We show that adding noise to data before making data public is effective at screening p-hacked findings: spurious explanations of the outcome variable produced by attempting multiple econometric specifications. Noise creates “baits” that affect two types of researchers differently. Uninformed p-hackers who engage in data mining with no prior information about the true causal mechanism often fall for baits and report verifiably wrong results when evaluated with the original data. But informed… 
1 Citations

Figures from this paper

Information Design for Differential Privacy
TLDR
It is shown that the simple geometric mechanism is optimal whenever data users face supermodular decision problems, and its solution is characterized.

References

SHOWING 1-10 OF 24 REFERENCES
Strategic Disclosure of Research Results: The Cost of Proving Your Honesty
PCT No. PCT/EP92/00856 Sec. 371 Date Jan. 4, 1993 Sec. 102(e) Date Jan. 4, 1993 PCT Filed Apr. 16, 1992 PCT Pub. No. WO92/19370 PCT Pub. Date Nov. 12, 1992.The invention relates to a thin-film
Persuasion Bias in Science: Can Economics Help?
The widespread adoption of randomized controlled experiments owes much to their ability to curtail researchers' conflicts of interest. This paper casts data collection and analysis in a
Strategic Sample Selection
TLDR
It is shown that anticipated selection locally reduces noise dispersion and thus increases informativeness if and only if the noise distribution is double logconvex, as with normal noise.
The reusable holdout: Preserving validity in adaptive data analysis
TLDR
A new approach for addressing the challenges of adaptivity based on insights from privacy-preserving data analysis is demonstrated, and how to safely reuse a holdout data set many times to validate the results of adaptively chosen analyses is shown.
P-Curve: A Key to the File Drawer
TLDR
By telling us whether the authors can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.
Bayesian Persuasion with Private Experimentation
This article studies a situation in which a sender tries to persuade a receiver by providing hard evidence that is generated by sequential private experimentation where the sender can design the
Evaluating replicability of laboratory experiments in economics
TLDR
To contribute data about replicability in economics, 18 studies published in the American Economic Review and the Quarterly Journal of Economics between 2011 and 2014 are replicated, finding that two-thirds of the 18 studies examined yielded replicable estimates of effect size and direction.
Research and the Approval Process: the Organization of Persuasion
An informer sequentially collects and disseminates information through costly research to persuade an evaluator to approve an activity. Payoffs and control rights are split between informer and
Why Most Published Research Findings Are False
TLDR
Simulations show that for most study designs and settings, it is more likely for a research claim to be false than true.
Estimating the reproducibility of psychological science
TLDR
A large-scale assessment suggests that experimental reproducibility in psychology leaves a lot to be desired, and correlational tests suggest that replication success was better predicted by the strength of original evidence than by characteristics of the original and replication teams.
...
...