• Corpus ID: 208637171

Another look at the Lady Tasting Tea and permutation-based randomization tests

  title={Another look at the Lady Tasting Tea and permutation-based randomization tests},
  author={Jesse Hemerik and Jelle J. Goeman},
  journal={arXiv: Methodology},
Fisher's famous Lady Tasting Tea experiment is often referred to as the first permutation test or as an example of such a test. Permutation tests are special cases of the general group invariance test. Recently it has been emphasized that the set of permutations used within a permutation test should have a group structure, in the algebraic sense. If not, the test can be very anti-conservative. In this paper, however, we note that in the Lady Tasting Tea experiment, the type I error rate is… 

Tables from this paper

Improving the Power of the Randomization Test
We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation


Optimal Permutation Tests for the Analysis of Group Randomized Trials
Two facts complicate the comparison of interventions in group randomized trials (GRT's), a family of clinical trials in which each member of a particular group receives the same treatment assignment.
The behaviour of some significance tests under experimental randomization
For the past 40 years, there has been confusion on the differences between tests of significance and tests of hypotheses to the point where data interpretation is presented as an accept-reject
Pros and cons of permutation tests in clinical trials.
  • V. Berger
  • Mathematics
    Statistics in medicine
  • 2000
It is concluded that permutation tests are ideal to study efficacy in a randomized clinical trial which compares, in a heterogeneous patient population, two or more treatments, each of which may be most effective in some patients, when the primary analysis does not adjust for covariates.
Rotation tests
This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression and offers an exact Monte Carlo solution to a classical problem of multiple testing.
Exact testing with random permutations
This paper provides an alternative proof, viewing the test as a “conditional Monte Carlo test” as it has been called in the literature, and results can be used to prove properties of various multiple testing procedures based on random permutations.
Randomization: The forgotten component of the randomized clinical trial.
How randomization-based inference can be used for virtually any outcome of interest in a clinical trial is described and very simple methods to rectify some of the oversight are described.
Permutation P-values Should Never Be Zero: Calculating Exact P-values When Permutations Are Randomly Drawn
  • B. Phipson, G. Smyth
  • Mathematics
    Statistical applications in genetics and molecular biology
  • 2010
It is argued that permutation should be viewed as generating an exact discrete null distribution, which is valid for any number of permutations and samples, and developed for exact p-values when permutations are randomly drawn.
Properties of the urn randomization in clinical trials.
Properties of simple randomization in clinical trials.
  • J. Lachin
  • Mathematics
    Controlled clinical trials
  • 1988