The earth is round (p < .05)

@article{Cohen1994TheEI,
  title={The earth is round (p < .05)},
  author={Jacob Cohen},
  journal={American Psychologist},
  year={1994},
  volume={49},
  pages={997-1003}
}
  • Jacob Cohen
  • Published 1 December 1994
  • Psychology
  • American Psychologist
After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews the problems with this practice, including near universal misinterpretation of p as the probability that H₀ is false, the misinterpretation that its complement is the probability of successful replication, and the mistaken assumption that if one rejects H₀ one thereby affirms the theory that led to the test… 
View via Publisher
agnrgroups.umd.edu
3,896 Citations
Highly Influential Citations
368
Background Citations
1,616
Methods Citations
497
Results Citations
89

3,896 Citations

The earth is flat (p > 0.05): significance thresholds and the crisis of unreplicable research

The widespread use of ‘statistical significance’ as a license for making a claim of a scientific finding leads to considerable distortion of the scientific process, and potential arguments against removing significance thresholds are discussed.

The Earth Is Not Round (p = .00)

Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This…

The Earth is spherical ( p < 0 : 05 ) : alternative methods of statistical inference

A literature review was conducted to understand the limitations of well-known statistical analysis techniques, particularly analysis of variance. The review is structured around six major points: (1)…

The Earth Is Not Round ( p 1⁄4 . 00 )

Continued discussion and debate regarding the appropriate use of null hypothesis significance testing (NHST) has led to greater reliance on effect size testing (EST) in published literature. This…

How significant (p < 0.05) is geomorphic research?

The pervasive application of the Null Hypothesis Significance Test in geomorphic research runs counter to widespread, long running, and often severe criticism of the method in the broader scientific…

Manipulating the Alpha Level Cannot Cure Significance Testing

We argue that making accept/reject decisions on scientific hypotheses, including a recent call for changing the canonical alpha level from p = 0.05 to p = 0.005, is deleterious for the finding of new…

Assessing environmentally significant effects: a better strength-of-evidence than a single P value?

A strength-of-evidence procedure that lends itself to a simple confidence interval interpretation and is accompanied by a strength- of-evidence matrix that has many desirable features: not only a strong/moderate/dubious/weak categorisation of the results, but also recommendations about the desirability of collecting further data to strengthen findings.

Non-significant results in ecology: a burden or a blessing in disguise?

To find out whether the statistical significance of the results affects the publication of ecological studies, the fate of manuscripts from Finnish and Swedish doctoral dissertations on ecological topics is followed up.

Sherlock Holmes and the Death of the Null Hypothesis

In the eighty years since R.A. Fisher’s original work, null hypothesis significance testing has become the ubiquitous research methodology in fields as diverse as biology, agronomy, social science,…

Is the call to abandon p-values the red herring of the replicability crisis?

Skepticism is expressed that alternative hypothesis testing frameworks, such as Bayes factors, are a solution to the replicability crisis and the value of applying the same standards of evidence that psychologists demand in choosing between competing substantive hypotheses is highlighted.
...

References

SHOWING 1-10 OF 46 REFERENCES

Theoretical risks and tabular asterisks: Sir Karl, Sir Ronald, and the slow progress of soft psychology.

Abstract Theories in “soft” areas of psychology lack the cumulative character of scientific knowledge. They tend neither to be refuted nor corroborated, but instead merely fade away as people lose…

Do studies of statistical power have an effect on the power of studies

The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal…

Statistical significance in psychological research.

  • D. Lykken
  • Psychology
    Psychological bulletin
  • 1968
Sapolsky (1964) developed the following substantive theory: Some psychiatric patients entertain an unconscious belief in the "cloacal theory of birth" which involves the notions of oral impregnation and anal parturition, which led Sapolsky to predict that Rorschach frog responders show.

Theory-Testing in Psychology and Physics: A Methodological Paradox

  • P. Meehl
  • Psychology
    Philosophy of Science
  • 1967
Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research,…

BELIEF IN THE LAW OF SMALL NUMBERS

“Suppose you have run an experiment on 20 subjects, and have obtained a significant result which confirms your theory ( z = 2.23, p If you feel that the probability is somewhere around .85, you may…

THINGS I HAVE LEARNED (SO FAR)

This is an account of what I have learned (so far) about the application of statistics to psychology and the other sociobiomedical sciences. It includes the principles "less is more" (fewer…

On the probability of making Type I errors.

A statistical test leads to a Type I error whenever it leads to the rejection of a null hypothesis that is in fact true. The probability of making a Type I error can be characterized in the following…

A Handbook for Data Analysis in the Behavioral Sciences: Statistical Issues

This book discusses methodological and statistical issues surrounding the development of Mathematical Models in Psychology, as well as some of the techniques used in Bayesian Statistics, a branch of statistics based on Bayesian inference.

A Handbook for data analysis in the behavioral sciences : methodological issues

This book discusses methodological and statistical issues surrounding the development of Mathematical Models in Psychology, as well as some of the techniques used in Bayesian Statistics, a branch of statistics based on Bayesian inference.

The Philosophy of Multiple Comparisons

'Abstract. This paper is based on the 1989 Miller Memorial Lecture at Stanford University. The topic was chosen because of Rupert Miller's long involvement and significant contributions to multiple…