Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial.

@article{Emerson2010TestingFT,
  title={Testing for the presence of positive-outcome bias in peer review: a randomized controlled trial.},
  author={Gwendolyn B. Emerson and Winston J Warme and Fredric M. Wolf and James D. Heckman and Richard A. Brand and Seth S Leopold},
  journal={Archives of internal medicine},
  year={2010},
  volume={170 21},
  pages={
          1934-9
        }
}
BACKGROUND If positive-outcome bias exists, it threatens the integrity of evidence-based medicine. METHODS We sought to determine whether positive-outcome bias is present during peer review by testing whether peer reviewers would (1) recommend publication of a "positive" version of a fabricated manuscript over an otherwise identical "no-difference" version, (2) identify more purposefully placed errors in the no-difference version, and (3) rate the "Methods" section in the positive version… 
The effects of industry funding and positive outcomes in the interpretation of clinical trial results: a randomized trial among Dutch psychiatrists
TLDR
In this study, industry funding disclosure was not associated with the perceived credibility nor judgement of clinical relevance of a fictional RCT by psychiatrists and positive study outcomes were found to be less credible compared to negative outcomes, but industry funding had no significant effects.
Role of Editorial and Peer Review Processes in Publication Bias: Analysis of Drug Trials Submitted to Eight Medical Journals
TLDR
There was no tendency to preferably publish manuscripts on drug RCTs that reported positive results, suggesting that publication bias may occur mainly prior to submission.
Impact of spin in the abstracts of articles reporting results of randomized controlled trials in the field of cancer: the SPIIN randomized controlled trial.
TLDR
Spin in abstracts can have an impact on clinicians' interpretation of the trial results and there was no statistically significant difference in the clinicians' rating of the importance of the study or the need to run another trial.
The perceived feasibility of methods to reduce publication bias
TLDR
Mandatory publication was generally favoured by respondents, and several underlying barriers to change emerged, including scientific culture, impact factors, and researcher training; these should be further explored to reduce publication bias.
Comment on: “Publication Bias, with a Focus on Psychiatry: Causes and Solutions”
TLDR
The author suggests that the best solution is a model similar to the approval process for the US Food and Drug Administration (FDA) which authors would submit protocols for journal peer-review before the study is conducted.
A two-step manuscript submission process can reduce publication bias.
  • Y. Smulders
  • Economics
    Journal of clinical epidemiology
  • 2013
A prospective study on an innovative online forum for peer reviewing of surgical science
TLDR
An open online forum for peer review is piloted at the British Journal of Surgery, but it attracts few reviews, of lower quality than conventional peer reviews.
Peer review comments on drug trials submitted to medical journals differ depending on sponsorship, results and acceptance: a retrospective cohort study
Objective During peer review, submitted manuscripts are scrutinised by independent experts to assist journal editors in their decision-making and to help improve the quality of articles. In this
Evaluation of an internal review process for grants and manuscripts in the Canadian Critical Care Trials Group.
TLDR
The present evaluation has assisted in understanding and improving the current internal review process and report a highly favourable evaluation of an existinginternal review process.
...
...

References

SHOWING 1-10 OF 33 REFERENCES
Publication bias and clinical trials.
Minimizing the three stages of publication bias.
TLDR
Bias can be minimized by insisting on high-quality research and thorough literature reviews, eliminating the double standard concerning peer review and informed consent applied to clinical research and practice, and publishing legitimate trials regardless of their results.
Bias towards publishing positive results in orthopedic and general surgery: a patient safety issue?
TLDR
There is a disproportionately high number of articles reporting positive results published in the surgical literature, and a bias towards publishing positive data will systematically overestimate the clinical relevance of treatment effects by disregarding important information derived from unpublished negative studies.
Positive-outcome bias and other limitations in the outcome of research abstracts submitted to a scientific meeting.
TLDR
Positive-outcome bias was evident when studies were submitted for consideration and was amplified in the selection of abstracts for both presentation and publication, neither of which was strongly related to study design or quality.
Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles.
TLDR
The reporting of trial outcomes is not only frequently incomplete but also biased and inconsistent with protocols and Published articles, as well as reviews that incorporate them, may therefore be unreliable and overestimate the benefits of an intervention.
Publication prejudices: An experimental study of confirmatory bias in the peer review system
  • M. Mahoney
  • Psychology
    Cognitive Therapy and Research
  • 2005
Confirmatory bias is the tendency to emphasize and believe experiences which support one's views and to ignore or discredit those which do not. The effects of this tendency have been repeatedly
Publication bias in orthopaedic research: an analysis of scientific factors associated with publication in the Journal of Bone and Joint Surgery (American Volume).
TLDR
Orthopaedic researchers should submit negative and neutral studies for publication, confident that the likelihood of acceptance will not be influenced by the direction of study findings, as positive and nonpositive studies were accepted at similar rates.
The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials.
TLDR
The revised CONSORT statement presented in this paper incorporates new evidence and addresses some criticisms of the original statement to improve the reporting of an RCT, enabling readers to understand a trial's conduct and to assess the validity of its results.
...
...