• DOI:10.24963/ijcai.2019/87
• Corpus ID: 49303175

On Strategyproof Conference Peer Review

@inproceedings{Xu2018OnSC,
  title={On Strategyproof Conference Peer Review},
  author={Yichong Xu and H. Zhao and Xiaofei Shi and Nihar B. Shah},
  booktitle={International Joint Conference on Artificial Intelligence},
  year={2018}
}

• Yichong Xu, H. Zhao, +1 author Nihar B. Shah
• Published in
  International Joint…
  16 June 2018
• Computer Science

We consider peer review under a conference setting where there are conflicts between the reviewers and the submissions. Under such conflicts, reviewers can manipulate their reviews in a strategic manner to influence the final rankings of their own papers. Present-day peer-review systems are not designed to guard against such strategic behavior, beyond minimal (and insufficient) checks such as not assigning a paper to a conflicted reviewer. In this work, we address this problem through the lens of social choice, and present a theoretical framework for strategyproof and efficient peer review. Given the conflict graph which satisfies a simple property, we first present and analyze a flexible framework for reviewer-assignment and aggregation for the reviews that guarantees not only strategyproofness but also a natural efficiency property (unanimity). Our framework is based on the so-called partitioning method, and can be treated as a generalization of this type of method to conference peer review settings. We then empirically show that the requisite property on the (authorship) conflict graph is indeed satisfied in the ICLR-17 submissions data, and further demonstrate a simple trick to make the partitioning method more practically appealing under conference peer-review settings. Finally, we complement our positive results with negative theoretical results where we prove that under slightly stronger requirements, it is impossible for any algorithm to be both strategyproof and efficient.