Mapping the Bid Behavior of Conference Referees

@article{Rodriguez2007MappingTB,
  title={Mapping the Bid Behavior of Conference Referees},
  author={Michael A. Rodriguez and Johan Bollen and Herbert Van de Sompel},
  journal={ArXiv},
  year={2007},
  volume={abs/cs/0605110}
}
Capitalizing on order effects in the bids of peer-reviewed conferences to secure reviews by expert referees
TLDR
It is shown that order effects lead to unconscious favoring of early-submitted papers to the detriment of later-Submitted papers, and it is advised to counterbalance order effects during the bidding phase of peer review by promoting the submissions with fewer bids to potential referees.
An algorithm to determine peer-reviewers
TLDR
An algorithm to automatically determine the most appropriate reviewers for a manuscript by way of a co-authorship network data structure and a relative-rank particle-swarm algorithm is presented.
A SUPER* Algorithm to Optimize Paper Bidding in Peer Review
TLDR
An algorithm called SUPER*, inspired by the A* algorithm, is presented, which considerably outperforms baselines deployed in existing systems, consistently reducing the number of papers with fewer than requisite bids by 50-75% or more, and is also robust to various real world complexities.
A Market-Inspired Bidding Scheme for Peer Review Paper Assignment
TLDR
It is shown that by assigning ‘budgets’ to reviewers and a ‘price’ for every paper that is (roughly) proportional to its demand, the best response of a reviewer is to bid sincerely, and match the budget even when it is not enforced.
AAAI 2020 Tutorial on Fairness and Bias in Peer Review and other Sociotechnical Intelligent Systems (Part II on Peer Review)
TLDR
This AAAI tutorial (part 2) discusses several problems, empirical studies, proposed solutions, and open problems in this domain of peer review.
PeerReview4All: Fair and Accurate Reviewer Assignment in Peer Review
TLDR
A fairness objective is to maximize the review quality of the most disadvantaged paper, in contrast to the commonly used objective of maximizing the total quality over all papers, and an assignment algorithm based on an incremental max-flow procedure is designed that is near-optimally fair.
TheWebConf 2020 Tutorial on Fairness and Bias in Peer Review and other Sociotechnical Intelligent Systems (Part II on Peer Review)
TLDR
This TheWebConf tutorial (part 2) discusses several problems, empirical studies, proposed solutions, and open problems in this domain.
Principled Methods to Improve Peer Review
TLDR
A survey of recent works towards developing an algorithmic toolkit for improved peer-review processes, which focuses on achieving objectives of fairness, accuracy, and robustness in these goals.
A Survey on Reviewer Assignment Problem
TLDR
This survey is not only to cover variations of RAP that have appeared in the literature, but also to identify the practical challenge and current progress for developing intelligent RAP systems.
...
...

References

SHOWING 1-10 OF 29 REFERENCES
Peer review. Crude and understudied, but indispensable.
PEER REVIEW is not perfect, and when it is done sloppily, journals publish research that is flawed. Even when peer review is rigorous, flawed research sometimes gets into the literature. Journals
The Conference Paper‐Reviewer Assignment Problem*
TLDR
A technique for finding a good assignment of papers to reviewers and an important feature of the solution is that each paper is sent to at least one reviewer who is “as expert as possible” for that paper.
The Influence of Peer Review on the Research Assessment Exercise
TLDR
It is concluded that because of the many criticisms of peer review, it is unwise to base funding decisions on second level peer review of articles that have already undergone peer review.
The peer‐review process
TLDR
Any scholarly publishing system will need to locate financial support to at least that extent, and a system of lump‐sum payment by the authors' funders is best placed to cover this cost while providing universal free access to scholarly material.
The Paradox of Peer Review: Admitting too Much or Allowing too Little?
Peer review – the means by which one's equals assess the quality of one's scholarly work – has been used to determine academic merit for more than three centuries. Although the results of academic
Scholarly communication and bibliometrics
TLDR
The Future of Bibliometrics and its Applications: A Case Study of Agricultural Research Within the European Community Core Journals of the Rapidly Changing Research Front of "Superconductivity" is reviewed.
Simulating Network Influence Algorithms Using Particle-Swarms: PageRank and PageRank-Priors
TLDR
This paper will describe a particular implementation of a particle-swarm that can simulate the behavior of the popular PageRank algorithm in both its global-rank and relative-rank incarnations.
Algorithms for estimating relative importance in networks
TLDR
This paper defines a general framework and a number of different algorithms, building on ideas from social networks, graph theory, Markov models, and Web graph analysis, which can be used to study relative importance in real-world networks including a network of interactions among September 11th terrorists.
...
...