• Corpus ID: 14647771

Publication Bias: The "File-Drawer" Problem in Scientific Inference

@article{Scargle1999PublicationBT,
  title={Publication Bias: The "File-Drawer" Problem in Scientific Inference},
  author={Jeffrey D. Scargle},
  journal={arXiv: Data Analysis, Statistics and Probability},
  year={1999}
}
  • J. Scargle
  • Published 17 September 1999
  • Economics
  • arXiv: Data Analysis, Statistics and Probability
Publication bias arises whenever the probability that a study is published depends on the statistical significance of its results. This bias, often called the file-drawer effect since the unpublished results are imagined to be tucked away in researchers' file cabinets, is potentially a severe impediment to combining the statistical results of studies collected from the literature. With almost any reasonable quantitative model for publication bias, only a small number of studies lost in the file… 

Figures from this paper

What Fuels Publication Bias?
Summary Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a
Beyond Publication Bias
This review considers several meta-regression and graphical methods that can differentiate genuine empirical effect from publication bias. Publication selection exists when editors, reviewers, or
Measuring the effects of publication bias in political science
Prior research finds that statistically significant results are overrepresented in scientific publications. If significant results are consistently favored in the review process, published results
Result-Blind Peer Reviews and Editorial Decisions A Missing Pillar of Scientific Culture
TLDR
It is argued that peer-reviewed journals based on the principle of rigorous evaluation of research proposals before results are known would address the persisting problem of publication bias more efficiently than other tools and techniques suggested so far.
Replication, statistical consistency, and publication bias.
Is Truth in A uthority or Authority in Truth? Limitations to the Publication of Scientific Research
Fisher J, Steele J. Is Truth in A uthority or Authority in T ruth? Limitations to the Publication of Scientific Research. JEPonline 2012;15(1):57-64. This paper examines the limitations and potential
A Generalized Publication Bias Model
Scargle (2000) has discussed Rosenthal and Rubin's (1978) fail-safe number” (FSN) method for estimating the number of unpublished studies in meta-analysis. He concluded that this FSN cannot possibly
Detecting and avoiding likely false‐positive findings – a practical guide
TLDR
It is argued that a culture of ‘you can publish if your study is rigorous’ creates a systematic bias against the null hypothesis which renders meta‐analyses questionable and may even lead to a situation where hypotheses become difficult to falsify.
The fault in our stars : Measuring and correcting significance bias in political science
Prior research finds that statistically significant results are overrepresented in scientific publications. If significant results are consistently favored in the review process, published results
Meta-Analyses with Binary Outcomes: How Many Studies Need to be Omitted to Detect a Publication Bias?
TLDR
Evidence indicates that standard methods for detection of a publication bias do not necessarily detect such a bias; thus, additional tests for publication bias need to be applied.
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 110 REFERENCES
Selection Models and the File Drawer Problem
TLDR
This paper uses selection models, or weighted distributions, to deal with one source of bias, namely the failure to report studies that do not yield statistically significant results, and applies selection models to two approaches that have been suggested for correcting the bias.
Publication Bias: The Problem That Won't Go Away
TLDR
Meta-analyses performed using data from four prospective investigations in which a total of 997 initiated studies were followed to learn of study results, publication status, and reasons for nonpublication indicate that there is a positive association between "significant" study results and publication.
How important is publication bias? A synthesis of available data.
  • K. Dickersin
  • Medicine
    AIDS education and prevention : official publication of the International Society for AIDS Education
  • 1997
TLDR
This article will review the major evidence available regarding publication bias and will suggest measures for overcoming the problem.
Diagoras of Melos (500 BC): an early analyst of publication bias
A Non-parametric "Trim and Fill" Method of Assessing Publication Bias in Meta-analysis
Meta-analysis collects and synthesizes results from individual studies to estimate an overall eeect size. If published studies are chosen, say through a literature review, an inherent selection bias
Practical estimates of the effect of publication bias in meta- analysis
TLDR
The 'trim and fill' method is presented, a new technique developed for estimating and adjusting for the numbers and outcomes of missing studies that has the advantage of being computationally simple and, in practical situations, to perform better than existing methods.
Publication bias in meta-analysis: A Bayesian data-augmentation approach to account for issues exemplified in the passive smoking debate - Comments and rejoinders
TLDR
A Bayesian approach is introduced which estimates and adjusts for publication bias in the passive smoking meta-analysis, and it is estimated that the estimated excess risk may be overstated by around 30%, both in U.S. studies and in the global collection of studies.
Review articles and publication bias.
TLDR
It is shown that publication bias is a serious problem which complicates the interpretation of reviews and meta-analyses and can only be avoided by registration of all trials before data collection is started.
Adjustment for Publication and Quality Bias in Bayesian Meta-analysis
Meta-analysis reviews, collects, and synthesizes individual sample surveys to estimate an overall eeect size. If the studies for a meta-analysis are chosen through a literature review, an inherent
File drawer problem
This dissertation provides normative, descriptive, and prescriptive analyses of a scientist’s decision to share data. The normative analysis (Chapter Two) concludes that, although there is no logical
...
1
2
3
4
5
...