The file drawer problem and tolerance for null results

@article{Rosenthal1979TheFD,
  title={The file drawer problem and tolerance for null results},
  author={Robert Rosenthal},
  journal={Psychological Bulletin},
  year={1979},
  volume={86},
  pages={638-641}
}
  • R. Rosenthal
  • Published 1 May 1979
  • Psychology
  • Psychological Bulletin
For any given research area, one cannot tell how many studies have been conducted but never reported. The extreme view of the "file drawer problem" is that journals are filled with the 5% of the studies that show Type I errors, while the file drawers are filled with the 95% of the studies that show nonsignificant results. Quantitative procedures for computing the tolerance for filed and future null results are reported and illustrated, and the implications are discussed. (15 ref) (PsycINFO… 
View via Publisher
pages.ucsd.edu
7,067 Citations
Highly Influential Citations
415
Background Citations
2,258
Methods Citations
1,089
Results Citations
426

7,067 Citations

Estimating the Effect of the File Drawer Problem in Meta-Analysis

The table shows that distortions exaggerating the effect size are substantial and that the exaggerations of effects are strongest when the true effect size approaches zero, meaning that a meta-analysis could be very misleading were thetrue effect size close to zero.

P-Curve: A Key to the File Drawer

By telling us whether the authors can rule out selective reporting as the sole explanation for a set of findings, p-curve offers a solution to the age-old inferential problems caused by file-drawers of failed studies and analyses.

A Novel Application of the File Drawer Formula: Rejected Study Bias in Meta-Analysis

A novel use of Orwin's 1983 formula is proposed that produces an estimate of the potential bias in the mean ES due to missing ESs from studies that were located and retrieved, but rejected because of insufficient data for ES calculation.

File Drawer Problem

  • J. Brand
  • Psychology
    The SAGE Encyclopedia of Research Design
  • 2022
This dissertation provides normative, descriptive, and prescriptive analyses of a scientist’s decision to share data. The normative analysis (Chapter Two) concludes that, although there is no logical…

Do Research Literatures Give Correct Answers?

The rationale for insisting on properly designed studies is to ensure the construction of research literatures that are not influenced by biases of any sort. Once a literature is established,…

Revisiting the file drawer problem in meta‐analysis: An assessment of published and nonpublished correlation matrices.

The file drawer problem rests on the assumption that statistically non-significant results are less likely to be published in primary-level studies and less likely to be included in meta-analytic…

Publication Bias: The "File-Drawer" Problem in Scientific Inference

Publication bias arises whenever the probability that a study is published depends on the statistical significance of its results. This bias, often called the file-drawer effect since the unpublished…

Selection Models and the File Drawer Problem

This paper uses selection models, or weighted distributions, to deal with one source of bias, namely the failure to report studies that do not yield statistically significant results, and applies selection models to two approaches that have been suggested for correcting the bias.

p-Curve and Effect Size

Journals tend to publish only statistically significant evidence, creating a scientific record that markedly overstates the size of effects. We provide a new tool that corrects for this bias without…

Documentation of the File Drawer Problem in Academic Finance Journals

The file drawer problem is a publication bias wherein editors of journals are much more likely to accept empirical papers with statistically significant results than those with nonsignificant…
...

References

SHOWING 1-10 OF 17 REFERENCES

Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa

Abstract There is some evidence that in fields where statistical tests of significance are commonly used, research which yields nonsignificant results is not published. Such research being unknown to…

Some Methods for Strengthening the Common χ 2 Tests

Since the x2 tests of goodness of fit and of association in contingency tables are presented in many courses on statistical methods for beginners in the subject, it is not surprising that x2 has…

Further Evidence for the Cliff Effect in the Interpretation of Levels of Significance

In a recent study by Rosenthal and Gaito (1963) a group of faculty members and a group of graduate students in psychology rated their degree of confidence or belief in 12 p values ranging from .001…

Meta-analysis of psychotherapy outcome studies.

Results of nearly 400 controlled evalua- tions of psychotherapy and counseling were coded and integrated statistically. The findings provide convincing evidence of the efficacy of psychotherapy. On…

Gender Effects in Decoding Nonverbal Cues

This article summarizes results of 7S studies that reported accuracy for males and females at decoding nonverbal communication. The following attributes of the studies were coded: year, sample size,…

The Handbook of Social Psychology

VOLUME 2. Part III: The Social World. 21. EVOLUTIONARY SOCIAL PSYCHOLOGY (Steven L. Neuberg, Douglas T. Kenrick, and Mark Schaller). 22. MORALITY (Jonathan Haidt and Selin Kesebir). 23. AGGRESSION…

The Interpretation of Levels of Significance by Psychological Researchers

Combining results of independent studies.

Experimenter effects in behavioral research, Enlarged ed.

Selected quantitative techniques and attitude measurement