Comparison of evidence of treatment effects in randomized and nonrandomized studies.

@article{Ioannidis2001ComparisonOE,
  title={Comparison of evidence of treatment effects in randomized and nonrandomized studies.},
  author={John P. A. Ioannidis and Anna-Bettina Haidich and M Pappa and Nikos Pantazis and Styliani Kokori and Maria G. Tektonidou and Despina G. Contopoulos-Ioannidis and Joseph Lau},
  journal={JAMA},
  year={2001},
  volume={286 7},
  pages={
          821-30
        }
}
CONTEXT There is substantial debate about whether the results of nonrandomized studies are consistent with the results of randomized controlled trials on the same topic. OBJECTIVES To compare results of randomized and nonrandomized studies that evaluated medical interventions and to examine characteristics that may explain discrepancies between randomized and nonrandomized studies. DATA SOURCES MEDLINE (1966-March 2000), the Cochrane Library (Issue 3, 2000), and major journals were searched… 

Figures and Tables from this paper

Comparison of Effects in Randomized Controlled Trials With Observational Studies in Digestive Surgery
TLDR
One fourth of observational studies gave different results than randomized trials, and between-study heterogeneity was more common in observational studies in the field of digestive surgery.
Beyond randomized controlled trials: A critical comparison of trials with nonrandomized studies
TLDR
This work reviewed the reasons for possible discrepancies between RCTs and observational studies and found both designs are susceptible to particular bias, so neither provides perfect information.
Comparison of Treatment Effect Estimates From Prospective Nonrandomized Studies With Propensity Score Analysis and Randomized Controlled Trials of Surgical Procedures
TLDR
Prospective NRSs with suitable and careful PS analysis can be relied upon as evidence when RCTs are not possible, and there was no statistically significant difference in treatment effect betweenNRSs with PS analysis and R CTs.
Meta-analytic comparison of randomized and nonrandomized studies of breast cancer surgery.
TLDR
Randomized controlled trials comparing surgical procedures for breast cancer may demonstrate clinically relevant differences in effect estimates in 20%-40% of cases relative to those generated by nRCTs, depending on which metric is used.
Comparison of evidence on harms of medical interventions in randomized and nonrandomized studies
TLDR
Evidence from randomized trials is compared with that from epidemiologic studies to determine whether they give different estimates of risk for important harms of medical interventions, and nonrandomized studies are often conservative in estimating absolute risks of harms.
Assessing observational studies of medical treatments
TLDR
Reporting of observational studies of medical treatments was often inadequate to compare study designs or allow other meaningful interpretation of results, so variations in treatment specifics, outcome definition or confounding were identified as possible causes.
Healthcare outcomes assessed with observational study designs compared with those assessed in randomized trials.
TLDR
A systematic analysis of study design features, risk of bias, parameter interpretation, and effect size for all types of randomized and non-experimental observational studies is needed to identify specific differences in design types and potential biases.
Meta-analyses including non-randomized studies of therapeutic interventions: a methodological review
TLDR
Some key methodological components of the systematic review process—search for grey literature, description of the type of NRSI included, assessment of risk of confounding bias and reporting of whether crude or adjusted estimates were combined—are not adequately carried out or reported in meta-analyses including NRSI.
Comparing Nonrandomized Observational Studies With Randomized Controlled Trials in Cervical Disc Arthroplasty: A Meta-analysis
TLDR
Prospective observational studies that utilize the same features of RCTs such as inclusion and exclusion criteria validated clinical outcomes, and statistical methods can provide valuable information about the treatment effects on a generalizable population.
...
...

References

SHOWING 1-10 OF 66 REFERENCES
A comparison of observational studies and randomized, controlled trials
  • Benson, Hartz
  • Medicine, Psychology
    American journal of ophthalmology
  • 2000
TLDR
Little evidence is found that estimates of treatment effects in observational studies reported after 1984 are either consistently larger than or qualitatively different from those obtained in randomized, controlled trials.
Randomized, controlled trials, observational studies, and the hierarchy of research designs.
TLDR
The results of well-designed observational studies (with either a cohort or a case-control design) do not systematically overestimate the magnitude of the effects of treatment as compared with those in randomized, controlled trials on the same topic.
Effect of the statistical significance of results on the time to completion and publication of randomized efficacy trials.
TLDR
There is a time lag in the publication of negative findings that occurs mostly after the completion of the trial follow-up among randomized efficacy trials.
Issues in comparisons between meta-analyses and large trials.
TLDR
To evaluate protocols comparing meta-analyses and large trials in order to understand if and why they disagree on the concordance of these 2 clinical research methods, systematic comparison of protocol designs, study selection, definitions of agreement, analysis methods, and reported discrepancies are evaluated.
How study design affects outcomes in comparisons of therapy. I: Medical.
TLDR
Any evaluation of an innovation may include both bias and the true efficacy of the new therapy, therefore any evaluation of a new therapy may consider making adjustments for the average bias associated with a study design.
Randomized versus historical controls for clinical trials.
The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials
TLDR
On average, failure to use randomisation or adequate concealment of allocation resulted in larger estimates of effect due to a poorer prognosis in non-randomlyselected control groups compared with randomly selected control groups.
Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials.
TLDR
Empirical evidence is provided that inadequate methodological approaches in controlled trials, particularly those representing poor allocation concealment, are associated with bias.
Randomized trials or observational tribulations?
TLDR
If claims that observational studies give results similar to those of randomized, controlled trials lead to more observational studies of therapeutic interventions and fewer randomized,controlled trials, there are considerable dangers to clinical research and even to the well-being of patients.
Publication bias in clinical research
...
...