# Another look at the Lady Tasting Tea and permutation-based randomization tests

@article{Hemerik2019AnotherLA, title={Another look at the Lady Tasting Tea and permutation-based randomization tests}, author={Jesse Hemerik and Jelle J. Goeman}, journal={arXiv: Methodology}, year={2019} }

Fisher's famous Lady Tasting Tea experiment is often referred to as the first permutation test or as an example of such a test. Permutation tests are special cases of the general group invariance test. Recently it has been emphasized that the set of permutations used within a permutation test should have a group structure, in the algebraic sense. If not, the test can be very anti-conservative. In this paper, however, we note that in the Lady Tasting Tea experiment, the type I error rate is…

## Tables from this paper

## One Citation

Improving the Power of the Randomization Test

- Mathematics
- 2020

We consider the problem of evaluating designs for a two-arm randomized experiment with the criterion being the power of the randomization test for the one-sided null hypothesis. Our evaluation…

## References

SHOWING 1-10 OF 55 REFERENCES

Optimal Permutation Tests for the Analysis of Group Randomized Trials

- Mathematics
- 2001

Two facts complicate the comparison of interventions in group randomized trials (GRT's), a family of clinical trials in which each member of a particular group receives the same treatment assignment.…

Revisiting Fisher’s ‘Lady Tasting Tea’ from a perspective of sensory discrimination testing

- Mathematics
- 2015

Abstract The Lady Tasting Tea is a famous real story in the history of development of statistics, related to R.A. Fisher, one of the greatest statisticians and founders of modern statistics. The main…

The behaviour of some significance tests under experimental randomization

- Mathematics
- 1969

For the past 40 years, there has been confusion on the differences between tests of significance and tests of hypotheses to the point where data interpretation is presented as an accept-reject…

Pros and cons of permutation tests in clinical trials.

- Mathematics, MedicineStatistics in medicine
- 2000

It is concluded that permutation tests are ideal to study efficacy in a randomized clinical trial which compares, in a heterogeneous patient population, two or more treatments, each of which may be most effective in some patients, when the primary analysis does not adjust for covariates.

Rotation tests

- Mathematics, Computer ScienceStat. Comput.
- 2005

This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression and offers an exact Monte Carlo solution to a classical problem of multiple testing.

Exact testing with random permutations

- Mathematics, Computer ScienceTest
- 2018

This paper provides an alternative proof, viewing the test as a “conditional Monte Carlo test” as it has been called in the literature, and results can be used to prove properties of various multiple testing procedures based on random permutations.

Randomization: The forgotten component of the randomized clinical trial.

- Medicine, PsychologyStatistics in medicine
- 2019

How randomization-based inference can be used for virtually any outcome of interest in a clinical trial is described and very simple methods to rectify some of the oversight are described.

Permutation P-values Should Never Be Zero: Calculating Exact P-values When Permutations Are Randomly Drawn

- Mathematics, MedicineStatistical applications in genetics and molecular biology
- 2010

It is argued that permutation should be viewed as generating an exact discrete null distribution, which is valid for any number of permutations and samples, and developed for exact p-values when permutations are randomly drawn.

Permutation and Bayesian tests for testing random effects in linear mixed-effects models.

- Computer Science, MedicineStatistics in medicine
- 2019

A permutation test based on the likelihood ratio test statistic, which can be easily used for testing multiple random effects and any subset of them in linear mixed-effects models is developed and compared with a Bayesian test developed using Bayes factors.

Properties of the urn randomization in clinical trials.

- Mathematics, MedicineControlled clinical trials
- 1988

The large-sample permutational null distributions of linear rank statistics for testing the equality of treatment groups based on the urn design are described and a combined covariate-adjusted test of treatment effect is described.