• Corpus ID: 208637171

Another look at the Lady Tasting Tea and permutation-based randomization tests

  title={Another look at the Lady Tasting Tea and permutation-based randomization tests},
  author={Jesse Hemerik and Jelle J. Goeman},
  journal={arXiv: Methodology},
Fisher's famous Lady Tasting Tea experiment is often referred to as the first permutation test or as an example of such a test. Permutation tests are special cases of the general group invariance test. Recently it has been emphasized that the set of permutations used within a permutation test should have a group structure, in the algebraic sense. If not, the test can be very anti-conservative. In this paper, however, we note that in the Lady Tasting Tea experiment, the type I error rate is… 

Tables from this paper

Improving the Power of the Randomization Test
We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation


Optimal Permutation Tests for the Analysis of Group Randomized Trials
Two facts complicate the comparison of interventions in group randomized trials (GRT's), a family of clinical trials in which each member of a particular group receives the same treatment assignment.
Revisiting Fisher’s ‘Lady Tasting Tea’ from a perspective of sensory discrimination testing
Abstract The Lady Tasting Tea is a famous real story in the history of development of statistics, related to R.A. Fisher, one of the greatest statisticians and founders of modern statistics. The main
The behaviour of some significance tests under experimental randomization
For the past 40 years, there has been confusion on the differences between tests of significance and tests of hypotheses to the point where data interpretation is presented as an accept-reject
Pros and cons of permutation tests in clinical trials.
  • V. Berger
  • Mathematics, Medicine
    Statistics in medicine
  • 2000
It is concluded that permutation tests are ideal to study efficacy in a randomized clinical trial which compares, in a heterogeneous patient population, two or more treatments, each of which may be most effective in some patients, when the primary analysis does not adjust for covariates.
Rotation tests
This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression and offers an exact Monte Carlo solution to a classical problem of multiple testing.
Exact testing with random permutations
This paper provides an alternative proof, viewing the test as a “conditional Monte Carlo test” as it has been called in the literature, and results can be used to prove properties of various multiple testing procedures based on random permutations.
Randomization: The forgotten component of the randomized clinical trial.
How randomization-based inference can be used for virtually any outcome of interest in a clinical trial is described and very simple methods to rectify some of the oversight are described.
Permutation P-values Should Never Be Zero: Calculating Exact P-values When Permutations Are Randomly Drawn
  • B. Phipson, G. Smyth
  • Mathematics, Medicine
    Statistical applications in genetics and molecular biology
  • 2010
It is argued that permutation should be viewed as generating an exact discrete null distribution, which is valid for any number of permutations and samples, and developed for exact p-values when permutations are randomly drawn.
Permutation and Bayesian tests for testing random effects in linear mixed-effects models.
A permutation test based on the likelihood ratio test statistic, which can be easily used for testing multiple random effects and any subset of them in linear mixed-effects models is developed and compared with a Bayesian test developed using Bayes factors.
Properties of the urn randomization in clinical trials.
The large-sample permutational null distributions of linear rank statistics for testing the equality of treatment groups based on the urn design are described and a combined covariate-adjusted test of treatment effect is described.