A sensible formulation of the significance test.

@article{Jones2000ASF,
  title={A sensible formulation of the significance test.},
  author={L. V. Jones and J. Tukey},
  journal={Psychological methods},
  year={2000},
  volume={5 4},
  pages={
          411-4
        }
}
The conventional procedure for null hypothesis significance testing has long been the target of appropriate criticism. A more reasonable alternative is proposed, one that not only avoids the unrealistic postulation of a null hypothesis but also, for a given parametric difference and a given error probability, is more likely to report the detection of that difference. 

Topics from this paper

Constrained Bayesian Methods for Testing Directional Hypotheses Restricted False Discovery Rates
The traditional formulation of testing simple basic hypothesis versus composite alternative is a well studied problem in many scientific works [1-8]. The problem of making the sense about directionExpand
On Confidence Intervals and Two-Sided Hypothesis Testing
This thesis consists of a summary and six papers, dealing with confidence intervals and two-sided tests of point-null hypotheses.In Paper I, we study Bayesian point-null hypothesis tests based on cExpand
The Significance Test Controversy Revisited
This chapter revisits the significance test controversy in the light of Jeffreys’ views about the role of statistical inference in experimental investigations. These views have been clearly expressedExpand
Undesirable optimality results in multiple testing?
A number of authors have considered the problem of making multiple comparisons among level-one parameters in multilevel models. This is a setting in which Bayesian procedures have a natural samplingExpand
A Five-Decision Testing Procedure to Infer the Value of a Unidimensional Parameter
ABSTRACT A statistical test can be seen as a procedure to produce a decision based on observed data, where some decisions consist of rejecting a hypothesis (yielding a significant result) and some doExpand
Beyond statistical inference: a decision theory for science.
  • P. Killeen
  • Medicine
  • Psychonomic bulletin & review
  • 2006
TLDR
The decision theory proposed here calculates the expected utility of an effect on the basis of the probability of replicating it and a utility function on its size, consistent with alternate measures of effect size, such as r2 and information transmission, and with Bayesian model selection criteria. Expand
Beyond statistical inference: A decision theory for science
Traditional null hypothesis significance testing does not yield the probability of the null or its alternative and, therefore, cannot logically ground scientific decisions. The decision theoryExpand
The Significance Test Controversy Revisited: The Fiducial Bayesian Alternative
Introduction.- Preamble - Frequentist and Bayesian Inference.- The Fisher, Neyman-Pearson and Jeffreys Views of Statistical Tests.- GHOST: An Officially Recommended Practice.- The Significance TestExpand
No-Decision Classification: An Alternative to Testing for Statistical Significance
This paper proposes a new statistical technique for deciding which of two theories is better supported by a given set of data while allowing for the possibility of drawing no conclusion at all.Expand
Constrained Bayesian Method for Testing the Directional Hypotheses
TLDR
The paper discusses the generalization of constrained Bayesian method (CBM) for arbitrary loss functions and its application for testing the directional hypotheses and the ratio among discovery rates and the Type III errors rate in CBM is considered. Expand
...
1
2
3
4
5
...

References

SHOWING 1-10 OF 30 REFERENCES
Testing the Approximate Validity of Statistical Hypotheses
The distinction between statistical significance and material significance in hypotheses testing is discussed. Modifications of the customary tests, in order to test for the absence of materialExpand
In praise of the null hypothesis statistical test.
Jacob Cohen (1994) raised a number of questions about the logic and information value of the null hypothesis statistical test (NHST). Specifically, he suggested that: (a) The NHST does not tell usExpand
The test of significance in psychological research.
  • D. Bakan
  • Psychology, Medicine
  • Psychological bulletin
  • 1966
The test of significance does not provide the information concerning psychological phenomena characteristically attributed to it; and a great deal of mischief has been associated with its use. TheExpand
The earth is round (p < .05)
After 4 decades of severe criticism, the ritual of null hypothesis significance testing (mechanical dichotomous decisions around a sacred .05 criterion) still persists. This article reviews theExpand
The fallacy of the null-hypothesis significance test.
TLDR
To the experimental scientist, statistical inference is a research instrument, a processing device by which unwieldy masses of raw data may be refined into a product more suitable for assimilation into the corpus of science, and in this lies both strength and weakness. Expand
Multiple Three-Decision Rules for Parametric Signs
Abstract Rules are considered for deciding, subject to a global error bound, whether each of several parameters is positive. In general, the decision “inconclusive data,” in addition to “positive”Expand
Controlling Error in Multiple Comparisons, with Examples from State-to-State Differences in Educational Achievement
Three alternative procedures to adjust significance levels for multiplicity are the traditional Bonferroni technique, a sequential Bonferroni technique developed by Hochberg (1988), and a sequentialExpand
Statistical significance in psychological research.
  • D. Lykken
  • Psychology, Medicine
  • Psychological bulletin
  • 1968
TLDR
Sapolsky (1964) developed the following substantive theory: Some psychiatric patients entertain an unconscious belief in the "cloacal theory of birth" which involves the notions of oral impregnation and anal parturition, which led Sapolsky to predict that Rorschach frog responders show. Expand
Theory-Testing in Psychology and Physics: A Methodological Paradox
  • P. Meehl
  • Psychology
  • Philosophy of Science
  • 1967
Because physical theories typically predict numerical values, an improvement in experimental precision reduces the tolerance range and hence increases corroborability. In most psychological research,Expand
The Philosophy of Multiple Comparisons
'Abstract. This paper is based on the 1989 Miller Memorial Lecture at Stanford University. The topic was chosen because of Rupert Miller's long involvement and significant contributions to multipleExpand
...
1
2
3
...