Publication prejudices: An experimental study of confirmatory bias in the peer review system

@article{Mahoney2005PublicationPA,
  title={Publication prejudices: An experimental study of confirmatory bias in the peer review system},
  author={Michael J. Mahoney},
  journal={Cognitive Therapy and Research},
  year={2005},
  volume={1},
  pages={161-175}
}
  • M. Mahoney
  • Published 1 June 1977
  • Psychology
  • Cognitive Therapy and Research
Confirmatory bias is the tendency to emphasize and believe experiences which support one's views and to ignore or discredit those which do not. The effects of this tendency have been repeatedly documented in clinical research. However, its ramifications for the behavior of scientists have yet to be adequately explored. For example, although publication is a critical element in determining the contribution and impact of scientific findings, little research attention has been devoted to the… 
View on Springer
gwern.net
608 Citations
Highly Influential Citations
23
Background Citations
288
Methods Citations
15
Results Citations
28

608 Citations

Citation Type

Citation Type

Expert bias in peer review
  • J. Phillips
  • Psychology
    Current medical research and opinion
  • 2011
TLDR
This editorial will attempt to shine a light on expert bias in peer review and offer solutions for overcoming the problem.
Preventing the ends from justifying the means: withholding results to address publication bias in peer-review
TLDR
BMC Psychology is launching a pilot to trial a new ‘results-free’ peer-review process, whereby editors and reviewers are blinded to the study’s results, initially assessing manuscripts on the scientific merits of the rationale and methods alone.
Metascience on Peer Review: Testing the Effects of a Study’s Originality and Statistical Significance in a Field Experiment
Peer review has become the gold standard in scientific publishing as a selection method and a refinement scheme for research reports. However, despite its pervasiveness and conferred importance,…
Introducing prospective manuscript review to address publication bias.
  • L. Mell, A. Zietman
  • Psychology
    International journal of radiation oncology, biology, physics
  • 2014
Cognitive Bias in the Peer Review Process: Understanding a Source of Friction between Reviewers and Researchers
TLDR
Evidence for a cognitive bias where respondents to a survey asking about the importance of particular validity and reliability method practices gave different answers depending on whether they were asked to answer the survey as a researcher or as a reviewer is found.
An Excess of Positive Results: Comparing the Standard Psychology Literature With Registered Reports
Selectively publishing results that support the tested hypotheses (“positive” results) distorts the available evidence for scientific claims. For the past decade, psychological scientists have been…
Null Effects and Publication Bias in Special Education Research
Researchers sometimes conduct a study and find that the predicted relation between variables did not exist or that the intervention did not have a positive impact on student outcomes; these are…
What Fuels Publication Bias?
Summary Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a…
Imperfect referees: Reducing the impact of multiple biases in peer review
TLDR
The findings show that the referees' static profiles are more dominant in peer review bias when compared to their dynamic behavioral context, and that the reliability of referees' judgments varies along their static profiles and is contingent on the temporal interval between 2 consecutive reviews.
Replication Research, Publication Bias, and Applied Behavior Analysis
TLDR
It is argued that contingencies of publication bias that led to the “replication crisis” also operate on applied behavior analysis researchers who use single-case research designs (SCRD) and that journals that publish SCRD research establish journal standards for publication of noneffect studies.
...
...

References

SHOWING 1-10 OF 27 REFERENCES
Stratification in American Science
Although differential ranking in science is not readily visible to lay observers, American science is, in fact, sharply graded. Rewards and facilities for research are concentrated among relatively…
Psychology of the scientist: An analysis of problem-solving bias
The problem-solving skills of 30 Ph.D. scientists were compared to those of 15 conservative Protestant ministers. Of particular interest was the frequency with which these groups generated…
The Academic Marketplace.
"This volume is a must for anyone interested in academic problems and will produce the emotion of recognition in those concerned, and the emotion of surprise in those outside the field."-Los Angeles…
Effects of Externally Mediated Rewards on Intrinsic Motivation.
Two laboratory experiments and one field experiment were conducted to investigate the effects of external rewards on intrinsic motivation to perform an activity. In each experiment, subjects were…
Undermining children's intrinsic interest with extrinsic reward: A test of the "overjustification" hypothesis.
A field experiment was conducted with children to test the "overjustification" hypothesis suggested by self-percepti on theory—the proposition that a person's intrinsic interest in an activity may be…
INTRINSIC MOTIVATION, EXTRINSIC REINFORCEMENT, AND INEQUITY
If a person who is intrinsically motivated to perform an activity begins to receive external reinforcement for the activity, what will happen to his intrinsic motivation? Previous studies and the…
Human memory
  • A. Parkin
  • Political Science
    Current Biology
  • 1999
Public Knowledge—The Social Dimension of Science
Criticism and the Growth of Knowledge
  • 1972
Memory And Attention
...
...