Statistical Procedures and the Justification of Knowledge in Psychological Science

@article{Rosnow1989StatisticalPA,
  title={Statistical Procedures and the Justification of Knowledge in Psychological Science},
  author={Ralph L. Rosnow and Robert Rosenthal},
  journal={American Psychologist},
  year={1989},
  volume={44},
  pages={1276-1284}
}
Justification, in the vernacular language of philosophy of science, refers to the evaluation, defense, and confirmation of claims of truth. In this article, we examine some aspects of the rhetoric of justification, which in part draws on statistical data analysis to shore up facts and inductive inferences. There are a number of problems of methodological spirit and substance that in the past have been resistant to attempts to correct them. The major problems are discussed, and readers are… 
View via Publisher
ist-socrates.berkeley.edu
884 Citations
Highly Influential Citations
52
Background Citations
266
Methods Citations
74
Results Citations
23

Tables from this paper

884 Citations

Citation Type

Citation Type

Statistical Significance Tests, Effect Size Reporting and the Vain Pursuit of Pseudo-Objectivity
Two themes are argued in this comment on the use of statistical significance tests. First, effect sizes are an important aspect of results that should be reported. However, 10 empirical studies (some…
From Statistical Significance to Effect Estimation: Statistical Reform in Psychology, Medicine and Ecology
Compelling criticisms of statistical significance testing (or Null Hypothesis Significance Testing, NHST) can be found in virtually all areas of the social and life sciences—including economics,…
The Spread of Statistical Significance Testing in Psychology
Because the widespread use of statistical significance testing has deleterious consequences for the development of a cumulative knowledge base, the American Psychological Association's Board of…
On the Logic (plus some history and philosophy) of Statistical Tests and Scientific Investigation
TLDR
This contribution is on elementary models connecting both levels that have been very popular in psychological research - statistical tests and an elegant contemporary treatment of the latter theory.
The statistical theories of Fisher and of Neyman and Pearson: A methodological perspective
Most of the debates around statistical testing suffer from a failure to identify clearly the features specific to the theories invented by Fisher and by Neyman and Pearson. These features are…
Misuse of statistical test in three decades of psychotherapy research.
This article reviews the misuse of statistical tests in psychotherapy research studies published in the Journal of Consulting and Clinical Psychology in the years 1967-1968, 1977-1978, and 1987-1988.…
Scientific versus Statistical Inference
Abstract We argue that the goals of scientists in data analysis and scientific communication do not match the logic of hypothesis testing as it is typically taught in introductory statistics courses.…
Best research practices in psychology: Illustrating epistemological and pragmatic considerations with the case of relationship science.
TLDR
This article contrasts the false-positives-reduction (FPR) approach with an alternative, the error balance (EB) approach, which argues that any serious consideration of optimal scientific practice must contend simultaneously with both false-positive and false-negative errors.
Quantitative science and the definition of measurement in psychology
^ge d that establishing quantitative science involves two research tasks: the scientific one of showing that the relevant attribute is quantitative; and the instrumental on& of constructing…
The (mis)reporting of statistical results in psychology journals
TLDR
The authors' results indicate that around 18% of statistical results in the psychological literature are incorrectly reported, and that errors were often in line with researchers’ expectations.
...
...

References

SHOWING 1-10 OF 90 REFERENCES
Evidence and scientific research.
This commentary reviews the arguments for and against the use of p-values put forward in the Journal and other forums, and shows that they are all missing both a measure and concept of "evidence."…
Statistical Power Analysis and Research Results.
I was most pleased by the recent publication by Brewer, "On the Power of Statistical Tests in the American Educational Research Journal" (1972), and understandably delighted with his heavy reliance,…
Social Psychology as History
An analysis of theory and research in social psychology reveals that while methods of research are scientific in character, theories of social behavior are primarily reflections of contemporary…
The Unaccountable Element in Science
I hope that it will become clear in the course of this paper what I mean by calling some elements of science unaccountable. Let me now say only that I shall speak of the contributions made to…
On the Power of Statistical Tests in the American Educational Research Journal 1
It is almost universally accepted by educational researchers that the power (probability of rejecting Ho when Ho is false, that is, 1-0-) of a statistical test is important and should be substantial.…
Publication Decisions and their Possible Effects on Inferences Drawn from Tests of Significance—or Vice Versa
Abstract There is some evidence that in fields where statistical tests of significance are commonly used, research which yields nonsignificant results is not published. Such research being unknown to…
Do studies of statistical power have an effect on the power of studies
The long-term impact of studies of statistical power is investigated using J. Cohen's (1962) pioneering work as an example. We argue that the impact is nil; the power of studies in the same journal…
Judgment Studies: Design, Analysis, and Meta-Analysis
Because of the complexity of human behaviour a great many research variables must be constructed from the building blocks of human judgement. A teacher's warmth, a psychotherapist's ability to create…
Meaning in Context: Is There Any Other Kind?
While the contextual grounding of the meanings of human action and language is vital to our everyday understanding of our own and others' behavior, the importance of context has been largely ignored…
THE POWER OF STATISTICAL TESTS
Abstract : The concept of power for statistical tests of hypotheses, and various ideas connected with it, are described and illustrated. The power is given for a number of the common statistical…
...
...