Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice

  title={Fearing the Future of Empirical Psychology: Bem's (2011) Evidence of Psi as a Case Study of Deficiencies in Modal Research Practice},
  author={Etienne P LeBel and Kurt R. Peters},
  journal={Review of General Psychology},
  pages={371 - 379}
In this methodological commentary, we use Bem's (2011) recent article reporting experimental evidence for psi as a case study for discussing important deficiencies in modal research practice in empirical psychology. We focus on (a) overemphasis on conceptual rather than close replication, (b) insufficient attention to verifying the soundness of measurement and experimental procedures, and (c) flawed implementation of null hypothesis significance testing. We argue that these deficiencies… Expand
Time to overcome the neglect of effect sizes in teaching psychological research findings.
Due to reliance on null-hypothesis significance testing, researchers had widely neglected effect size (ES) in their reporting of psychological research results. The American Psychological AssociationExpand
Surveillance Is Not the Answer, and Replication Is Not a Test: Comment on Kepes and McDaniel, “How Trustworthy Is the Scientific Literature in I–O Psychology?”
Although we share Kepes and McDaniel’s (2013) concern about the state of affairs in I–O psychology, we think their emphasis on control and correction will, in the end, be counterproductive.Expand
A New Replication Norm for Psychology
In recent years, there has been a growing concern regarding the replicability of findings in psychology, including a mounting number of prominent findings that have failed to replicate viaExpand
Heterogeneity of research results: New perspectives on psychological science
Replicability of research findings is a key issue in psychology, and has been the subject of considerable discussion by researchers. Replication is crucial to scientific principles, and underpins theExpand
- Error Control to Validity Proper : Problems With a Short-Sighted False-Positive
Several influential publications have sensitized the community of behavioral scientists to the dangers of inflated effects and false-positive errors leading to the unwarranted publication ofExpand
Understanding Replication in a Way That Is True to Science
  • B. Haig
  • Psychology
  • Review of General Psychology
  • 2021
In this article, I critically examine a number of widely held beliefs about the nature of replication and its place in science, with particular reference to psychology. In doing so, I present aExpand
Toward a Brighter Future for Psychology as an Observation Oriented Science
Observation Oriented Modeling is presented as an alternative approach toward data conceptualization and analysis for the social and life sciences that is rooted in philosophical realism and an attitude toward data analysis centered around causality and common sense. Expand
Registered reports: an early example and analysis
A retrospective and observational comparison of Registered and non-Registered Reports published in the EJP during a seventeen-year period provides circumstantial evidence to suggest that the approach helped to reduce questionable research practices. Expand
RUNNING HEAD: How trustworthy is the scientific literature in I-O psychology? How trustworthy is the scientific literature in I-O psychology?
The trustworthiness of research findings has been questioned in many domains of science. This paper calls for a review of the trustworthiness of the scientific literature in I-O psychology and aExpand
Conceptualizing and evaluating the replication of research results
Abstract Many recent discussions have focused on the role of replication in psychological science. In this article, we examine three key issues in evaluating the conclusions that follow from resultsExpand


Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011).
It is concluded that Bem's p values do not indicate evidence in favor of precognition; instead, they indicate that experimental psychologists need to change the way they conduct their experiments and analyze their data. Expand
Gergen Versus the Mainstream: Are Hypotheses in Social Psychology Subject to Empirical Test?
K. J. Gergen's (1982) argument that hypotheses in social psychology are not empirical propositions is critically examined and shown to be erroneous. Nevertheless, this article demonstrates that,Expand
Effect sizes and p values: what should be reported and what should be replicated?
The most-criticized flaws of NHT can be avoided when the importance of a hypothesis is used to determine that a finding is worthy of report, and when p approximately equal to .05 is treated as insufficient basis for confidence in the replicability of an isolated non-null finding. Expand
Statistical significance in psychological research.
  • D. Lykken
  • Psychology, Medicine
  • Psychological bulletin
  • 1968
Sapolsky (1964) developed the following substantive theory: Some psychiatric patients entertain an unconscious belief in the "cloacal theory of birth" which involves the notions of oral impregnation and anal parturition, which led Sapolsky to predict that Rorschach frog responders show. Expand
Quantitative science and the definition of measurement in psychology
^ge d that establishing quantitative science involves two research tasks: the scientific one of showing that the relevant attribute is quantitative; and the instrumental on& of constructingExpand
The persistence of underpowered studies in psychological research: causes, consequences, and remedies.
  • S. Maxwell
  • Psychology, Medicine
  • Psychological methods
  • 2004
Underpowered studies persist in the psychological literature and the effects on efforts to create a cumulative science are examined and the "curse of multiplicities" plays a central role. Expand
A Handbook for data analysis in the behavioral sciences : methodological issues
This book discusses methodological and statistical issues surrounding the development of Mathematical Models in Psychology, as well as some of the techniques used in Bayesian Statistics, a branch of statistics based on Bayesian inference. Expand
Reporting and Interpreting Research in PSPB: Practices, Principles, and Pragmatics
Recommendations are developed that are hoped will enhance the reporting of quantitative research in social and personality psychology and emphasize ways to increase transparency in research reports. Expand
The test of significance in psychological research.
  • D. Bakan
  • Medicine, Psychology
  • Psychological bulletin
  • 1966
The test of significance does not provide the information concerning psychological phenomena characteristically attributed to it; and a great deal of mischief has been associated with its use. TheExpand
Statistical Significance and Replicability
In spite of arguments to the contrary, psychologists, it is shown here, believe statistical significance (SS) signifies that a finding will replicate. The most visible argument that SS is not anExpand