The Power of Bias in Economics Research

@article{Ioannidis2017ThePO,
  title={The Power of Bias in Economics Research},
  author={John P. A. Ioannidis and T. D. Stanley and Hristos Doucouliagos},
  journal={Econometrics: Econometric Model Construction},
  year={2017}
}
We investigate two critical dimensions of the credibility of empirical economics research: statistical power and bias. We survey 159 empirical economics literatures that draw upon 64,076 estimates of economic parameters reported in more than 6,700 empirical studies. Half of the research areas have nearly 90% of their results under‐powered. The median statistical power is 18%, or less. A simple weighted average of those reported results that are adequately powered (power ≥ 80%) reveals that… Expand
The strategies of economic research - An empirical study
The paper analyzes the structure of strategies of economic research on a sample of all regular research papers in 10 general interest journals every 5th year from 1997 to 2017. It is 3,415 papers,Expand
The Influence of Hidden Researcher Decisions in Applied Microeconomics
Researchers make hundreds of decisions about data collection, preparation, and analysis in their research. We use a many-analysts approach to measure the extent and impact of these decisions. TwoExpand
Do consumers really follow a rule of thumb? Three thousand estimates from 144 studies say “probably not”
Abstract We show that three factors combine to explain the mean magnitude of excess sensitivity reported in studies estimating the consumption response to income changes: the use of macro data,Expand
To Replicate or Not to Replicate? Exploring Reproducibility in Economics Through the Lens of a Model and a Pilot Study
The sciences are in an era of an alleged ‘credibility crisis’. In this study, we discuss the reproducibility of empirical results, focusing on economics research. By combining theory and empiricalExpand
What Meta-Analyses Reveal About the Replicability of Psychological Research
TLDR
The low power and high heterogeneity that the survey finds fully explain recent difficulties to replicate highly regarded psychological studies and reveal challenges for scientific progress in psychology. Expand
Practical Significance, Meta-Analysis and the Credibility of Economics
TLDR
Through Monte Carlo simulation, it is found that large biases and high rates of false positives will often be found by conventional meta-analysis methods, and the routine application of meta-regression analysis and considerations of practical significance largely restore research credibility. Expand
Feature—Is a Replicability Crisis on the Horizon for Environmental and Resource Economics?
Environmental and resource economists pride themselves on the credibility of their empirical research. In other disciplines, however, the credibility of empirical research is increasingly beingExpand
Biases in Fiscal Multiplier Estimates
The "true" size of fiscal multipliers is widely debated by economists and policy makers as large (small) multipliers provide arguments to expand (cut) public spending. Within a meta-analyticalExpand
Methods Matter: P-Hacking and Causal Inference in Economics
The economics 'credibility revolution' has promoted the identification of causal relationships using difference-in-differences (DID), instrumental variables (IV), randomized control trials (RCT) andExpand
Biases in fiscal multiplier estimates
The "true" size of fiscal multipliers is widely debated by economists and policy makers as large (small) multipliers provide arguments to expand (cut) public spending. Within a meta-analyticalExpand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 135 REFERENCES
What's to Know About the Credibility of Empirical Economics?
The scientific credibility of economics is itself a scientific question that can be addressed with both theoretical speculations and empirical data. In this review, we examine the major parametersExpand
Are All Economic Facts Greatly Exaggerated? Theory Competition and Selectivity
is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a truncated distribution of results. The aim of this paper is to explore theExpand
The Standard Error of Regressions
Statistical significance as used in economics has weak theoretical justification. In particular it merges statistical and substantive significance. The 182 papers using regression analysis in theExpand
Replications in Economics: A Progress Report
This study reports on various aspects of replication research in economics. It includes (i) a brief history of data sharing and replication; (ii) the results of the authors’ survey administered toExpand
A Meta-Analysis of B-Convergence: The Legendary 2%
The topic of convergence is at the heart of a wide-ranging debate in the growth literature, and empirical studies of convergence differ widely in their theoretical backgrounds, empiricalExpand
To Replicate or Not to Replicate? Exploring Reproducibility in Economics Through the Lens of a Model and a Pilot Study
The sciences are in an era of an alleged ‘credibility crisis’. In this study, we discuss the reproducibility of empirical results, focusing on economics research. By combining theory and empiricalExpand
Theory competition and selectivity: are all economic facts greatly exaggerated?
There is growing concern and mounting evidence of selectivity in empirical economics. Most empirical economic literatures have a skewed (or truncated) distribution of results. The aim of this paperExpand
Size Matters: The Standard Error of Regressions in the American Economic Review
Significance testing as used has no theoretical justification. Our article in the Journal of Economic Literature (1996) showed that of the 182 full-length papers published in the 1980s in theExpand
Are estimates of the value of a statistical life exaggerated?
TLDR
This study demonstrates that correcting for selection bias is essential for studies that employ hedonic wage equations to estimate VSL, and identifies several sources for the wide heterogeneity found among reported VSL estimates. Expand
Publication Selection and the Income Elasticity of the Value of a Statistical Life
TLDR
The income elasticity of value of a statistical life is clearly and robustly inelastic, with a value of approximately 0.25-0.63, and there is clear evidence of the importance of controlling for levels of risk, differential publication selection bias, and the greater income sensitivity of VSL from stated preference surveys. Expand
...
1
2
3
4
5
...