“Repeated sampling from the same population?” A critique of Neyman and Pearson’s responses to Fisher

@article{Rubin2020RepeatedSF,
  title={“Repeated sampling from the same population?” A critique of Neyman and Pearson’s responses to Fisher},
  author={Mark Rubin},
  journal={European Journal for Philosophy of Science},
  year={2020},
  volume={10},
  pages={1-15}
}
  • Mark Rubin
  • Published 1 September 2020
  • Psychology
  • European Journal for Philosophy of Science
Fisher ( 1945a , 1945b , 1955 , 1956 , 1960 ) criticised the Neyman-Pearson approach to hypothesis testing by arguing that it relies on the assumption of “repeated sampling from the same population.” The present article considers the responses to this criticism provided by Pearson ( 1947 ) and Neyman ( 1977 ). Pearson interpreted alpha levels in relation to imaginary replications of the original test. This interpretation is appropriate when test users are sure that their replications will be… 
Applying Perspectival Realism to Frequentist Statistics: The Case of Jerzy Neyman’s Methodology and Philosophy
: I investigate the extent to which perspectival realism (PR) agrees with frequentist statistical methodology and philosophy, with an emphasis on J. Neyman’s views. Based on the example of the
Data quality, experimental artifacts, and the reactivity of the psychological subject matter
  • Uljana Feest
  • Philosophy
    European Journal for Philosophy of Science
  • 2022
While the term “reactivity” has come to be associated with specific phenomena in the social sciences, having to do with subjects’ awareness of being studied, this paper takes a broader stance on this
Assessing the Global and Local Uncertainty of Scientific Evidence in the Presence of Model Misspecification
TLDR
Non-parametric bootstrap methodologies for estimating the sampling distribution of the evidence estimator under model misspecification are developed, which allows us to determine how secure the authors are in their evidential statement.

References

SHOWING 1-10 OF 37 REFERENCES
What type of Type I error? Contrasting the Neyman–Pearson and Fisherian approaches in the context of exact and direct replications
TLDR
It is concluded that the replication crisis may be partly (not wholly) due to researchers’ unrealistic expectations about replicability based on their consideration of the Neyman–Pearson Type I error rate across a long run of exact replications.
The Alleged Crisis and the Illusion of Exact Replication
  • W. Stroebe, F. Strack
  • Psychology
    Perspectives on psychological science : a journal of the Association for Psychological Science
  • 2014
TLDR
It is proposed that for meaningful replications, attempts at reinstating the original circumstances are not sufficient and replicators must ascertain that conditions are realized that reflect the theoretical variable(s) manipulated (and/or measured) in the original study.
Final Collapse of the Neyman-Pearson Decision Theoretic Framework and Rise of the neoFisherian
This essay grew out of an examination of one-tailed significance testing. One-tailed tests were little advocated by the founders of modern statistics but are widely used and recommended nowadays in
What is replication?
TLDR
It is proposed that replication is a study for which any outcome would be considered diagnostic evidence about a claim from prior research, which reduces emphasis on operational characteristics of the study and increases emphasis on the interpretation of possible outcomes.
A tutorial on testing hypotheses using the Bayes factor.
TLDR
After reading this tutorial and executing the associated code, researchers will be able to use their own data for the evaluation of hypotheses by means of the Bayes factor, not only in thecontext of ANOVA models, but also in the context of other statistical models.
Alphabet Soup
Confusion over the reporting and interpretation of results of commonly employed classical statistical tests is recorded in a sample of 1,645 papers from 12 psychology journals for the period 1990
Making replication mainstream
TLDR
There are no theoretical or statistical obstacles to making direct replication a routine aspect of psychological science, and the need for an integrative summary of replication studies is addressed.
An Evaluation of Four Solutions to the Forking Paths Problem: Adjusted Alpha, Preregistration, Sensitivity Analyses, and Abandoning the Neyman-Pearson Approach
Gelman and Loken (2013, 2014) proposed that when researchers base their statistical analyses on the idiosyncratic characteristics of a specific sample (e.g., a nonlinear transformation of a variable
Errors in Statistical Inference Under Model Misspecification: Evidence, Hypothesis Testing, and AIC
TLDR
This work approximate analytically and numerically the performance of Neyman-Pearson hypothesis testing, Fisher significance testing, information criteria, and evidential statistics and shows that the evidence function concept fulfills the seeming objectives of model selection in ecology.
Psychology, Science, and Knowledge Construction: Broadening Perspectives from the Replication Crisis
TLDR
It is recommended that researchers adopt open science conventions of preregi‐stration and full disclosure and that replication efforts be based on multiple studies rather than on a single replication attempt.
...
...