Testing exchangeability: Fork-convexity, supermartingales and e-processes

@article{Ramdas2022TestingEF,
  title={Testing exchangeability: Fork-convexity, supermartingales and e-processes},
  author={Aaditya Ramdas and Johannes Ruf and Martin Larsson and Wouter M. Koolen},
  journal={Int. J. Approx. Reason.},
  year={2022},
  volume={141},
  pages={83-109}
}

Figures from this paper

Game-theoretic Formulations of Sequential Nonparametric One- and Two-Sample Tests

TLDR
The sequential testing framework constructed in this paper is versatile, and a discussion on applying these ideas to two related problems: testing for higher-order stochastic dominance, and testing for symmetry is discussed.

Anytime-valid sequential testing for elicitable functionals via supermartingales

: We design sequential tests for a large class of nonparametric null hypotheses based on elicitable and identifiable functionals. Such functionals are defined in terms of scoring functions and

A composite generalization of Ville's martingale theorem

We provide a composite version of Ville’s theorem that an event has zero measure if and only if there exists a martingale which explodes to infinity when that event occurs. This is a classic result

Estimating means of bounded random variables by betting

TLDR
A general approach for deriving concentration bounds that can be seen as a generalization and improvement of the celebrated Chernoff method is presented, based on a class of composite nonnegative martingales, with strong connections to testing by betting and the method of mixtures.

Comparing Sequential Forecasters

TLDR
A sequential inference procedure for estimating the time-varying difference in forecast quality as measured by any scoring rule, and can sequentially test a weak null hypothesis about whether one forecaster outperforms another on average over time.

Locally private nonparametric confidence intervals and sequences

TLDR
A nonparametric and sequentially interactive generalization of Warner’s famous “randomized response” mechanism is introduced, satisfying LDP for arbitrary bounded random variables, and CIs and CSs are provided for their means given access to the resulting privatized observations.

Interactive rank testing by betting

In order to test if a treatment is perceptibly different from a placebo in a randomized experiment with covariates, classical nonparametric tests based on ranks of observations/residuals have been

Log-optimal anytime-valid E-values

A unified framework for bandit multiple testing

TLDR
This work proposes a unified, modular framework for bandit FDR control that emphasizes the decoupling of exploration and summarization of evidence, and recovers matching sample complexity guarantees in this special case, and performs comparably or better in practice.

E-Statistics, Group Invariance and Anytime Valid Testing

We study worst-case growth-rate optimal (GROW) e -statistics for hypothesis testing between two group models. If the underlying group G acts freely on the observation space, there exists a maximally

References

SHOWING 1-10 OF 65 REFERENCES

Time-uniform Chernoff bounds via nonnegative supermartingales

We develop a class of exponential bounds for the probability that a martingale sequence crosses a time-dependent linear threshold. Our key insight is that it is both natural and fruitful to formulate

Admissible anytime-valid sequential inference must rely on nonnegative martingales.

TLDR
This work shows that martingales are also universal---all constructions of (composite) anytime $p-values, confidence sequences, or e-values must necessarily utilize nonnegative martingale, and provides several sophisticated examples, with special focus on the nonparametric problem of testing if a distribution is symmetric, where the new constructions render past methods inadmissible.

Sequential Estimation of Convex Divergences using Reverse Submartingales and Exchangeable Filtrations

We present a unified technique for sequential estimation of convex divergences between distributions, including integral probability metrics like the kernel maximum mean discrepancy, φ-divergences

Time-uniform, nonparametric, nonasymptotic confidence sequences

A confidence sequence is a sequence of confidence intervals that is uniformly valid over an unbounded time horizon. Our work develops confidence sequences whose widths go to zero, with nonasymptotic

Safe Testing

TLDR
Sharing Fisherian, Neymanian and Jeffreys-Bayesian interpretations, S-values and safe tests may provide a methodology acceptable to adherents of all three schools.

Estimating means of bounded random variables by betting

TLDR
A general approach for deriving concentration bounds that can be seen as a generalization and improvement of the celebrated Chernoff method is presented, based on a class of composite nonnegative martingales, with strong connections to testing by betting and the method of mixtures.

The Structure of m–Stable Sets and in Particular of the Set of Risk Neutral Measures

The study of dynamic coherent risk measures and risk adjusted values as introduced by Artzner, Delbaen, Eber, Heath and Ku, leads to a property called fork convexity. We give necessary and sufficient

Nonparametric Iterated-Logarithm Extensions of the Sequential Generalized Likelihood Ratio Test

TLDR
A nonparametric extension of the sequential generalized likelihood ratio (GLR) test and corresponding time-uniform confidence sequences for the mean of a univariate distribution and a flexible and practical method to construct time- uniform confidence sequence that are easily tunable to be uniformly close to the pointwise Chernoff bound over any target time interval are presented.

Testing Randomness Online

  • V. Vovk
  • Mathematics
    Statistical Science
  • 2021
The hypothesis of randomness is fundamental in statistical machine learning and in many areas of nonparametric statistics; it says that the observations are assumed to be independent and coming from

Mixture Martingales Revisited with Applications to Sequential Tests and Confidence Intervals

TLDR
New deviation inequalities that are valid uniformly in time under adaptive sampling in a multi-armed bandit model are presented, allowing us to analyze stopping rules based on generalized likelihood ratios for a large class of sequential identification problems, and to construct tight confidence intervals for some functions of the means of the arms.
...