A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications

@inproceedings{Kang2018ADO,
  title={A Dataset of Peer Reviews (PeerRead): Collection, Insights and NLP Applications},
  author={Dongyeop Kang and Waleed Ammar and Bhavana Dalvi and Madeleine van Zuylen and Sebastian Kohlmeier and Eduard H. Hovy and Roy Schwartz},
  booktitle={NAACL-HLT},
  year={2018}
}
Peer reviewing is a central component in the scientific publishing process. We present the first public dataset of scientific peer reviews available for research purposes (PeerRead v1) providing an opportunity to study this important artifact. The dataset consists of 14.7K paper drafts and the corresponding accept/reject decisions in top-tier venues including ACL, NIPS and ICLR. The dataset also includes 10.7K textual peer reviews written by experts for a subset of the papers. We describe the… CONTINUE READING
37
Twitter Mentions

Figures, Tables, Results, and Topics from this paper.

Key Quantitative Results

  • In the first task, we show that simple models can predict whether a paper is accepted with up to 21% error reduction compared to the majority baseline.
  • Our best model outperforms the majority classifier in all cases, with up to 22% error reduction.

Citations

Publications citing this paper.
SHOWING 1-10 OF 13 CITATIONS

Does My Rebuttal Matter? Insights from a Major NLP Conference

VIEW 5 EXCERPTS
CITES METHODS, BACKGROUND & RESULTS
HIGHLY INFLUENCED

A Sentiment Augmented Deep Architecture to Predict Peer Review Outcomes

  • 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL)
  • 2019
VIEW 7 EXCERPTS
CITES METHODS & RESULTS
HIGHLY INFLUENCED

Earlier Isn't Always Better: Sub-aspect Analysis on Corpus and System Biases in Summarization

Taehee Jung, Dongyeop Kang, Lucas Mentch, Eduard Hovy
  • ArXiv
  • 2019
VIEW 4 EXCERPTS
CITES BACKGROUND
HIGHLY INFLUENCED

A Joint Model for Multimodal Document Quality Assessment

  • 2019 ACM/IEEE Joint Conference on Digital Libraries (JCDL)
  • 2019
VIEW 2 EXCERPTS
CITES METHODS