Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice

@article{LeBel2011FearingTF,
  title={Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice},
  author={Etienne P LeBel and Kurt R. Peters},
  journal={Review of General Psychology},
  year={2011},
  volume={15},
  pages={371 - 379}
}
In this methodological commentary, we use Bem's (2011) recent article reporting experimental evidence for psi as a case study for discussing important deficiencies in modal research practice in empirical psychology. We focus on (a) overemphasis on conceptual rather than close replication, (b) insufficient attention to verifying the soundness of measurement and experimental procedures, and (c) flawed implementation of null hypothesis significance testing. We argue that these deficiencies… 

Surveillance Is Not the Answer, and Replication Is Not a Test: Comment on Kepes and McDaniel, “How Trustworthy Is the Scientific Literature in I–O Psychology?”

Although we share Kepes and McDaniel’s (2013) concern about the state of affairs in I–O psychology, we think their emphasis on control and correction will, in the end, be counterproductive.

Heterogeneity of research results: New perspectives on psychological science

Replicability of research findings is a key issue in psychology, and has been the subject of considerable discussion by researchers. Replication is crucial to scientific principles, and underpins the

- Error Control to Validity Proper : Problems With a Short-Sighted False-Positive

Several influential publications have sensitized the community of behavioral scientists to the dangers of inflated effects and false-positive errors leading to the unwarranted publication of

Understanding Replication in a Way That Is True to Science

  • B. Haig
  • Biology
    Review of General Psychology
  • 2020
It is maintained that viewing replication as just one element of the wide array of scientific endeavors leads to the conclusion that it is not as prominent in science as is often claimed.

Toward a Brighter Future for Psychology as an Observation Oriented Science

Observation Oriented Modeling is presented as an alternative approach toward data conceptualization and analysis for the social and life sciences that is rooted in philosophical realism and an attitude toward data analysis centered around causality and common sense.

Registered reports: an early example and analysis

A retrospective and observational comparison of Registered and non-Registered Reports published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices.

RUNNING HEAD: How trustworthy is the scientific literature in I-O psychology? How trustworthy is the scientific literature in I-O psychology?

The trustworthiness of research findings has been questioned in many domains of science. This paper calls for a review of the trustworthiness of the scientific literature in I-O psychology and a

From crisis of evidence to a “crisis” of relevance? Incentive-based answers for social psychology’s perennial relevance worries

Current controversies in social psychology have sparked the promotion of new rules for evidence in the field. This “crisis of evidence” echoes prior concerns from the 1970’s “crisis of social

The State of Social and Personality Science: Rotten to the Core, Not So Bad, Getting Better, or Getting Worse?

Perceptions of the current state of the field are more pessimistic than optimistic; the discussion has increased researchers’ intentions to avoid QRPs and adopt proposed best practices, and the estimated replicability of research published in 2003–2004 may not be as bad as many feared.
...

References

SHOWING 1-10 OF 72 REFERENCES

Gergen Versus the Mainstream: Are Hypotheses in Social Psychology Subject to Empirical Test?

K. J. Gergen's (1982) argument that hypotheses in social psychology are not empirical propositions is critically examined and shown to be erroneous. Nevertheless, this article demonstrates that,

Effect sizes and p values: what should be reported and what should be replicated?

The most-criticized flaws of NHT can be avoided when the importance of a hypothesis is used to determine that a finding is worthy of report, and when p approximately equal to .05 is treated as insufficient basis for confidence in the replicability of an isolated non-null finding.

Statistical significance in psychological research.

  • D. Lykken
  • Psychology
    Psychological bulletin
  • 1968
Sapolsky (1964) developed the following substantive theory: Some psychiatric patients entertain an unconscious belief in the "cloacal theory of birth" which involves the notions of oral impregnation and anal parturition, which led Sapolsky to predict that Rorschach frog responders show.

Quantitative science and the definition of measurement in psychology

^ge d that establishing quantitative science involves two research tasks: the scientific one of showing that the relevant attribute is quantitative; and the instrumental on& of constructing

The persistence of underpowered studies in psychological research: causes, consequences, and remedies.

Underpowered studies persist in the psychological literature and the effects on efforts to create a cumulative science are examined and the "curse of multiplicities" plays a central role.

A Handbook for data analysis in the behavioral sciences : methodological issues

This book discusses methodological and statistical issues surrounding the development of Mathematical Models in Psychology, as well as some of the techniques used in Bayesian Statistics, a branch of statistics based on Bayesian inference.

Reporting and Interpreting Research in PSPB: Practices, Principles, and Pragmatics

Recommendations are developed that are hoped will enhance the reporting of quantitative research in social and personality psychology and emphasize ways to increase transparency in research reports.

The test of significance in psychological research.

  • D. Bakan
  • Psychology
    Psychological bulletin
  • 1966
The test of significance does not provide the information concerning psychological phenomena characteristically attributed to it; and a great deal of mischief has been associated with its use. The

Statistical Significance and Replicability

In spite of arguments to the contrary, psychologists, it is shown here, believe statistical significance (SS) signifies that a finding will replicate. The most visible argument that SS is not an

Sexy But Often Unreliable: The Impact of Unreliability on the Replicability of Experimental Findings With Implicit Measures

It is demonstrated that a higher level of unreliability in such dependent variables is associated with substantially lower levels of replicability, which implies that this overlooked consequence can have far-reaching repercussions for the development of a cumulative science.
...