The p‐value statement, five years on

  title={The p‐value statement, five years on},
  author={Robert Matthews},
The American Statistical Association's 2016 p‐value statement generated debates and disagreements, editorials and symposia, and a plethora of ideas for how science could be changed for the better. Now, five years on, Robert Matthews asks what, if anything, has the statement achieved? 
Results Blind Science Publishing and a Decision-Theoretic Approach to Publishing
  • J. Locascio
  • Psychology
    Basic and Applied Social Psychology
  • 2022
Abstract In this paper, I revisit my earlier proposal for Results Blind Publishing (RBP) and have added some new perspectives and qualifications regarding it. RBP is a suggestion that research
Publication Policies for Replicable Research and the Community-Wide False Discovery Rate
This article demonstrates that a statistic called the local false discovery rate (lfdr), which incorporates this information, is a sufficient summary for addressing false positive rates.
Inferring the COVID-19 IFR with a simple Bayesian evidence synthesis of seroprevalence study data and imprecise mortality data
The results suggest that, despite immense efforts made to better understand the COVID-19 IFR, there remains a large amount of uncertainty and unexplained heterogeneity surrounding this important statistic.
Underdispersion in the reported Covid-19 case and death numbers may suggest data manipulations
We suggest a statistical test for underdispersion in the reported Covid-19 case and death numbers, compared to the variance expected under the Poisson distribution. Screening all countries in the
Variations in Definitions of Evidence-Based Interventions for Behavioral Health in Eight Selected U.S. States.
The variations in EBI-related terminology across states and within states, coupled with a lack of elaboration on the meaning of important terms and the predominant use of external rather than internal guidelines, may be a source of confusion for behavioral health provider agencies that seek direction about what constitutes an EBI.
Major sex differences in migraine prevalence among occupational categories: a cross-sectional study using UK Biobank
Gender-specific differences in the prevalence of migraine across a broad spectrum of occupational categories are scrutinized, shedding light on associations with important job-related features such as shift work, job satisfaction, and physical activity.
COVID-19 pandemic in Saint Petersburg, Russia: combining surveillance and population-based serological study data in May, 2020 - April, 2021
This study summarises results from four consecutive serological surveys conducted between May 2020 and April 2021 at St.Petersburg, Russia and combines them with other SARS-CoV-2 surveillance data to provide a comprehensive pandemic picture.
Major sex differences in migraine prevalence among occupational categories: a cross-sectional study using UK Biobank
Gender-specific differences in the prevalence of migraine across a broad spectrum of occupational categories show that migraine is genderdependently associated with physically demanding jobs and shift working.
Understanding Statistical Significance and Avoiding Common Pitfalls.


The ASA's p‐value statement, one year on
Its aim was to stop the misuse of statistical significance testing. But Robert Matthews argues that little has changed in the 12 months since the ASA's intervention
Justify your alpha
In response to recommendations to redefine statistical significance to P ≤ 0.005, we propose that researchers should transparently report and justify all choices they make when designing a study,
Redefine statistical significance
The default P-value threshold for statistical significance is proposed to be changed from 0.05 to 0.005 for claims of new discoveries in order to reduce uncertainty in the number of discoveries.
The ASA Statement on p-Values: Context, Process, and Purpose
Cobb’s concern was a long-worrisome circularity in the sociology of science based on the use of bright lines such as p< 0.05: “We teach it because it’s what we do; we do it because it’s what we
It’s time to talk about ditching statistical significance
  • Education
  • 2019
Looking beyond a much used and abused measure would make science harder, but better, and help scientists understand the world around them better.
Scientists rise up against statistical significance
Valentin Amrhein, Sander Greenland, Blake McShane and more than 800 signatories call for an end to hyped claims and the dismissal of possibly crucial effects.Valentin Amrhein, Sander Greenland, Blake
The p-Value Requires Context, Not a Threshold
Abstract It is widely recognized by statisticians, though not as widely by other researchers, that the p-value cannot be interpreted in isolation, but rather must be considered in the context of
Retire statistical significance
When was the last time you heard a seminar speaker claim there was ‘no difference’ between two groups because the difference was ‘statistically non-significant’? If your experience matches ours,
Valid P-Values Behave Exactly as They Should: Some Misleading Criticisms of P-Values and Their Resolution With S-Values
Abstract The present note explores sources of misplaced criticisms of P-values, such as conflicting definitions of “significance levels” and “P-values” in authoritative sources, and the consequent
The assessment of intrinsic credibility and a new argument for p < 0.005
  • L. Held
  • Philosophy
    Royal Society Open Science
  • 2019
An application to data from the Open Science Collaboration study on the reproducibility of psychological science suggests that intrinsic credibility of the original experiment is better suited to predict the success of a replication experiment than standard significance.