Peer Review: From “Sacred Ideals” to “Profane Realities”

  title={Peer Review: From “Sacred Ideals” to “Profane Realities”},
  author={David Ross Johnson and Joseph C. Hermanowicz},
Peer review, a socially structured process of evaluating scholarly and scientific performance, is a ubiquitous condition of role performance in the professoriate and central to the production of knowledge. Focusing on the evaluation of publication, this chapter directs attention to three features of peer review: its functional ideals and relationship with the academic reward system; the social organizational basis of peer review and trends that constrain it; and dysfunctions that arise related… Expand
Peer Review as an Evolving Response to Organizational Constraint: Evidence from Sociology Journals, 1952–2018
Double-blind peer review is a central feature of the editorial model of most journals in sociology and neighboring social scientific fields, yet there is little history of how and when its mainExpand
Honest Evaluation in the Academy
Honesty is widely understood as an ethical imperative in science and scholarship. This article examines the operation of this ethic in an area crucial to academe but which has not received sufficientExpand
Contemporary Academic Publishing: Democratization and Differentiation in Careers
ABSTRACT This study examined how publishing figures in the careers of academics as a means to study the social organization of higher education. Publishing is a means by which academics legitimateExpand
Criteria for assessing grant applications: a systematic review
This systematic review identifies and synthesizes studies that examine grant peer review criteria in an empirical and inductive manner and finds that some of the criteria peers use to evaluate grant applications do not conform to the fairness doctrine and the ideal of impartiality. Expand
Criteria for assessing grant applications: a systematic review
Criteria are an essential component of any procedure for assessing merit. Yet, little is known about the criteria peers use to assess grant applications. In this systematic review we thereforeExpand
Bifurcated Conversations in Sociological Studies of Religion and Gender
Feminist sociologists claim that while feminist insights have been incorporated in sociological paradigms and women sociologists have been well-integrated into academia, sociological frameworks haveExpand
What Are Reviewers Looking For? A Qualitative Analysis of Open-Ended Responses from a Questionnaire Sent to Faculty in Agricultural Communications
While peer review is the best system available for assessing the quality of research manuscripts, the system is imperfect at best. How peer review is conducted is often guided by unwritten rules,Expand
Co-existing Notions of Research Quality: A Framework to Study Context-specific Understandings of Good Research
Notions of research quality are contextual in many respects: they vary between fields of research, between review contexts and between policy contexts. Yet, the role of these co-existing notions inExpand
Do peers share the same criteria for assessing grant applications?
This study examines a basic assumption of peer review, namely, the idea that there is a consensus on evaluation criteria among peers, which is a necessary condition for the reliability of peer judgements, and identifies two consensus classes, two consensus-close classes, and a consensus-far class. Expand


Experience with NIH Peer Review: Researchers' Cynicism and Desire for Change
In the United States, peer review is central to the process by which many government agencies select research proposals for funding.' Although several different agency versions of peer review areExpand
Bias in peer review
This review provides a brief description of the function, history, and scope of peer review, and characterizes and examines the empirical, methodological, and normative claims of bias in peer review research; and assesses possible alternatives to the status quo. Expand
Peer-review practices of psychological journals: The fate of published articles, submitted again
A growing interest in and concern about the adequacy and fairness of modern peer-review practices in publication and funding are apparent across a wide range of scientific disciplines. AlthoughExpand
What is Originality in the Humanities and the Social Sciences?
Drawing on interviews with peer-review panelists from five multidisciplinary fellowship competitions, this paper analyzes one of the main criteria used to evaluate scholarship in the humanities andExpand
Editorial Judgments
Based on participant observation of editors’ decisions for a sociology journal, the paper investigates the peer review process. It shows a hidden interactivity in peer review, which is overlookedExpand
Publication prejudices: An experimental study of confirmatory bias in the peer review system
  • M. Mahoney
  • Psychology
  • Cognitive Therapy and Research
  • 2005
Confirmatory bias is the tendency to emphasize and believe experiences which support one's views and to ignore or discredit those which do not. The effects of this tendency have been repeatedlyExpand
Frame Search and Re-Search: How Quantitative Sociological Articles Change During Peer Review
Peer review is a central institution in academic publishing, yet its processes and effects on research remain opaque. Empirical studies have (1) been rare because data on the peer review process areExpand
Measuring the effectiveness of scientific gatekeeping
The research suggests that evaluative strategies that increase the mean quality of published science may also increase the risk of rejecting unconventional or outstanding work, and raises concerns regarding whether peer review is ill-suited to recognize and gestate the most impactful ideas and research. Expand
Social Biases and Solutions for Procedural Objectivity
An empirically sensitive formulation of the norms of transformative criticism must recognize that even public and shared standards of evaluation can be implemented in ways that unintentionallyExpand
What do we know about peer review?
  • S. Wessely
  • Psychology, Medicine
  • Psychological medicine
  • 1996
Recent research has identified several shortcomings in the peer review process, including the problem of referee reliability and how reliably reviewers make these assessments. Expand