A Kuhnian Critique of Psychometric Research on Peer Review

@article{Lee2012AKC,
  title={A Kuhnian Critique of Psychometric Research on Peer Review},
  author={Carole J. Lee},
  journal={Philosophy of Science},
  year={2012},
  volume={79},
  pages={859-870}
}
Psychometrically oriented researchers construe low interrater reliability measures for expert peer reviewers as damning for the practice of peer review. I argue that this perspective overlooks different forms of normatively appropriate disagreement among reviewers. Of special interest are Kuhnian questions about the extent to which variance in reviewer ratings can be accounted for by normatively appropriate disagreements about how to interpret and apply evaluative criteria within disciplines… Expand

Tables from this paper

Bias in peer review
A Social Epistemological Inquiry into Biases in Journal Peer Review
Commensuration Bias in Peer Review
The Social Structure of Consensus in Scientific Review
Ensuring the Quality, Fairness, and Integrity of Journal Peer Review: A Possible Role of Editors
Is Peer Review a Good Idea
Impact from the Evaluators’ Eye
...
1
2
3
...

References

SHOWING 1-10 OF 48 REFERENCES
Improving the peer-review process for grant applications: reliability, validity, bias, and generalizability.
Bias in peer review
Neglected considerations in the analysis of agreement among journal referees
Editors' requests of peer reviewers: a study and a proposal.
...
1
2
3
4
5
...