Reviewer bias in single- versus double-blind peer review

@article{Tomkins2017ReviewerBI,
  title={Reviewer bias in single- versus double-blind peer review},
  author={Andrew Tomkins and M. Zhang and William D. Heavlin},
  journal={Proceedings of the National Academy of Sciences of the United States of America},
  year={2017},
  volume={114},
  pages={12708 - 12713}
}
Peer review may be "single-blind," in which reviewers are aware of the names and affiliations of paper authors, or "double-blind," in which this information is hidden. [] Key Method We present a controlled experiment in which four committee members review each paper. Two of these four reviewers are drawn from a pool of committee members with access to author information; the other two are drawn from a disjoint pool without such access. This information asymmetry persists through the process of bidding for…

Figures and Tables from this paper

Does double‐blind peer review reduce bias? Evidence from a top computer science conference
TLDR
It is shown that double-blind peer review may have improved the quality of the selections by limiting other (non-author-prestige) biases, and an apparently unrelated change – the change of rating scale from 10 to 4 points – likely reduced prestige bias significantly, to an extent that affected papers’ acceptance.
Double-Blind Reviews: A Step Toward Eliminating Unconscious Bias
Starting in January 2022, we are excited to announce that Clinical and Translational Gastroenterology (CTG) will transition from a single-blind to a double-blind peer review system. The peer review
The cases for and against double-blind reviews
TLDR
It is concluded, at least for these journals, that double-blind review no longer benefits female authors and the pros and cons of the double- blind reviewing process are discussed.
Importance of the peer review process in scientific publications - proposed way of working for a new journal
Typically, in scientific publications, peer review processes are used to maintain the quality standards of the research activity and its results presented in manuscripts submitted for publication, to
Peer Review Bias: A Critical Review
Uptake and outcome of manuscripts in Nature journals by review model and author characteristics
TLDR
The proportion of authors that choose double-blind review is higher when they submit to more prestigious journals, they are affiliated with less prestigious institutions, or they are from specific countries; the double- blind option is also linked to less successful editorial outcomes.
Peer review: single-blind, double-blind, or all the way-blind?
  • T. Bazi
  • Psychology
    International Urogynecology Journal
  • 2019
A scholarly peer review is the process whereby referees scrutinize research work or a manuscript within their field of expertise and decide on its acceptability for publication in a journal or
Citations Beyond Self Citations: Identifying Authors, Affiliations, and Nationalities in Scientific Papers
TLDR
This work trains models that identify the authors, their affiliations, and their nationalities through real-world, large-scale experiments on the Microsoft Academic Graph, including the cold start scenario, and shows, against conventional belief, that the self-citations are no more informative than looking at the common citations.
Cite-seeing and Reviewing: A Study on Citation Bias in Peer Review
Citations play an important role in researchers’ careers as a key factor in evaluation of scientific impact. Many anecdotes advice authors to exploit this fact and cite prospective reviewers to try
Metrics and methods in the evaluation of prestige bias in peer review: A case study in computer systems conferences
The integrity of peer review is essential for modern science. Numerous studies have therefore focused on identifying, quantifying, and mitigating biases in peer review. One of these better-known
...
...

References

SHOWING 1-10 OF 34 REFERENCES
Improving publication quality by reducing bias with double-blind reviewing and author response
TLDR
The purpose of double-blind reviewing is to focus the evaluation process on the quality of the submission by reducing human biases with respect to the authors’ reputation, gender, and institution, by not revealing those details.
The myth of the double-blind review?: author identification using only citations
TLDR
For the KDD Cup 2003 competition's "Open Task," it was examined how well various automatic matching techniques could identify authors within the competition's very large archive of research papers.
Editorial: Single- versus double-blind reviewing
TLDR
This editorial analyzes from a variety of perspectives the controversial issue of single-blind versus double- Blind reviewing, and proposes a double-blind policy for TODS that attempts to minimize the costs while retaining the core benefit of fairness that double- blind reviewing provides.
Blind Peer Review by Academic Journals
Emerging trends in peer review—a survey
TLDR
Two major trends are identified: the rapidly expanding role of preprint servers that dispense with traditional peer review altogether, and the growth of “non-selective review,” focusing on papers' scientific quality rather than their perceived importance and novelty.
Double-blind reviewing: more placebo than miracle cure?
TLDR
It is believed that DBR mainly increases the perceived fairness of the reviewing process, but that may be an important benefit, and rather than waiting until the final stages, the review process needs to explicitly address the issue of workshop publications early on.
Impact of double-blind reviewing on SIGMOD publication rates
Starting with the 2001 SIGMOD conference, the SIGMOD Chair, in consultation with the SIGMOD Advisory Committee, imposed a double blind rule on all future SIGMOD conferences. While there are many
Double-blind reviewing at EvoLang 11 reveals gender bias
The impact of introducing double-blind reviewing in the most recent Evolution of Language conference is assessed. The ranking of papers is compared between EvoLang 11 (double-blind review) and
...
...