Bias in peer review

@article{Lee2013BiasIP,
  title={Bias in peer review},
  author={Carole J. Lee and C. Sugimoto and Guo Zhang and B. Cronin},
  journal={J. Assoc. Inf. Sci. Technol.},
  year={2013},
  volume={64},
  pages={2-17}
}
Research on bias in peer review examines scholarly communication and funding processes to assess the epistemic and social legitimacy of the mechanisms by which knowledge communities vet and self-regulate their work. Despite vocal concerns, a closer look at the empirical and methodological limitations of research on bias raises questions about the existence and extent of many hypothesized forms of bias. In addition, the notion of bias is predicated on an implicit ideal that, once articulated… Expand
View via Publisher
faculty.washington.edu
512 Citations
Highly Influential Citations
31
Background Citations
190
Methods Citations
12
Results Citations
9

512 Citations

Citation Type

Citation Type

The limitations to our understanding of peer review
TLDR
This work analyses peer review to assess where the major gaps in theoretical and empirical understanding of it lie, and identifies core themes including editorial responsibility, the subjectivity and bias of reviewers, the function and quality of peer review, and the social and epistemic implications ofpeer review. Expand
Peer Review: From “Sacred Ideals” to “Profane Realities”
Peer review, a socially structured process of evaluating scholarly and scientific performance, is a ubiquitous condition of role performance in the professoriate and central to the production of… Expand
Imperfect referees: Reducing the impact of multiple biases in peer review
TLDR
The findings show that the referees' static profiles are more dominant in peer review bias when compared to their dynamic behavioral context, and that the reliability of referees' judgments varies along their static profiles and is contingent on the temporal interval between 2 consecutive reviews. Expand
Cognitive Bias in the Peer Review Process: Understanding a Source of Friction between Reviewers and Researchers
TLDR
Evidence for a cognitive bias where respondents to a survey asking about the importance of particular validity and reliability method practices gave different answers depending on whether they were asked to answer the survey as a researcher or as a reviewer is found. Expand
Bias and truth in science evaluation: a simulation model of grant review panel discussions
Research funding organizations draw upon the expertise of peer review panels to decide which research proposals to fund. That of review panels is a collective task of information acquisition that is… Expand
Bias in peer review: a case study.
TLDR
Analysis of three datasets providing information on the attributes of authors and reviewers and review outcomes finds no evidence of bias in terms of gender, or the language or prestige of author and reviewer institutions in any of the three datasets, but some weak evidence of regional bias in all three. Expand
Peer review and gender bias: A study on 145 scholarly journals
TLDR
Findings suggest that peer review and editorial processes do not penalize manuscripts by women, and increasing gender diversity in editorial teams and referee pools could help journals inform potential authors about their attention to these factors and so stimulate participation by women. Expand
The Social and Psychological Costs of Peer Review
Research has evolved into a high-stake competition for journal space. This study examines the effects of peer rejections on individual scholars. I propose a transactional framework that organizes… Expand
The sociology of scientific validity: How professional networks shape judgement in peer review
Professional connections between the creators and evaluators of scientific work are ubiquitous, and the possibility of bias ever-present. Although connections have been shown to bias predictions of… Expand
Unprofessional peer reviews disproportionately harm underrepresented groups in STEM
TLDR
The results indicate that unprofessional reviews likely have and will continue to perpetuate the gap in STEM fields for traditionally underrepresented groups in the sciences. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 216 REFERENCES
Improving the peer-review process for grant applications: reliability, validity, bias, and generalizability.
TLDR
The authors propose a new approach, the reader system, which they evaluated with psychology and education grant proposals and found to be substantially more reliable and strategically advantageous than traditional peer reviews of grant applications. Expand
Gender differences in peer reviews of grant applications: A substantive-methodological synergy in support of the null hypothesis model
Peer review serves a gatekeeper role, the final arbiter of what is valued in academia, but is widely criticized in terms of potential biases—particularly in relation to gender. In this… Expand
Experience with NIH Peer Review: Researchers' Cynicism and Desire for Change
In the United States, peer review is central to the process by which many government agencies select research proposals for funding.' Although several different agency versions of peer review are… Expand
Editorial Judgments
Based on participant observation of editors’ decisions for a sociology journal, the paper investigates the peer review process. It shows a hidden interactivity in peer review, which is overlooked… Expand
Rebound peer review: a viable recourse for aggrieved authors?
  • C. Sen
  • Medicine
  • Antioxidants & redox signaling
  • 2012
TLDR
The rebound peer review track is a two-tier process that represents a hybrid of partially blinded and open peer review systems to make sure that every author has the opportunity to rescue their rejected work which they feel may have been victimized by the glitches of the current peer review system. Expand
Publication prejudices: An experimental study of confirmatory bias in the peer review system
  • M. Mahoney
  • Psychology
  • Cognitive Therapy and Research
  • 2005
Confirmatory bias is the tendency to emphasize and believe experiences which support one's views and to ignore or discredit those which do not. The effects of this tendency have been repeatedly… Expand
The effectiveness of the peer review process: inter-referee agreement and predictive validity of manuscript refereeing at Angewandte Chemie.
TLDR
The quality of peer review at Angewandte Chemie is examined according to the criteria of reliability (agreement among referees) and predictive validity, using an optimized study design and modern methods of statistical data analysis. Expand
Social Biases and Solutions for Procedural Objectivity
An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionally… Expand
Peer-review practices of psychological journals: The fate of published articles, submitted again
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. Although… Expand
Conservatism and risk-taking in peer review: Emerging ERC practices
This article explores the newly founded European Research Council's (ERC) peer review system and its ability to sustain its mission to promote excellent, groundbreaking research. The article explores… Expand
...
1
2
3
4
5
...