Universal inference

  title={Universal inference},
  author={Larry A. Wasserman and Aaditya Ramdas and Sivaraman Balakrishnan},
  journal={Proceedings of the National Academy of Sciences},
  pages={16880 - 16890}
Significance Most statistical methods rely on certain mathematical conditions, known as regularity assumptions, to ensure their validity. Without these conditions, statistical quantities like P values and confidence intervals might not be valid. In this paper we give a surprisingly simple method for producing statistical significance statements without any regularity conditions. The resulting hypothesis tests can be used for any parametric model and for several nonparametric models. We propose… 

Figures from this paper

Likelihood-Free Frequentist Inference: Confidence Sets with Correct Conditional Coverage

A practical procedure for the Neyman construction of confidence sets with nominal coverage, and diagnostics that estimate conditional coverage over the entire parameter space are presented.

Anytime Valid Tests of Conditional Independence Under Model-X

A sequential, anytime-valid method to test the conditional independence of a response Y and a predictor X given a random vector Z, based on e -statistics and test martingales, which generalize likelihood ratios and allow valid inference at arbitrary stopping times is proposed.

Confidence Intervals for Nonparametric Empirical Bayes Analysis

Abstract In an empirical Bayes analysis, we use data from repeated sampling to imitate inferences made by an oracle Bayesian with extensive knowledge of the data-generating distribution. Existing

Universal Inference Meets Random Projections: A Scalable Test for Log-concavity

This work finds that the highest power is obtained by using random projections to convert the d-dimensional testing problem into many one-dimensional problems, leading to a simple procedure that is statistically and computationally efficient.

Nonparametric Likelihood Ratio Test for Univariate Shape-constrained Densities

We provide a comprehensive study of a nonparametric likelihood ratio test on whether a random sample follows a distribution in a prespecified class of shapeconstrained densities. While the

Derandomized knockoffs: leveraging e-values for false discovery rate control

This paper introduces a methodology for derandomizing model-X knockoffs with provable FDR control, and proves that the derandomized procedure controls the FDR at the desired level, without any additional conditions.

E-values as unnormalized weights in multiple testing

This work shows that standard weighted multiple testing methods are not required when the weights are not constants, but are themselves e-values obtained from independent data, which could result in a massive increase in power.

Post-selection inference for e-value based confidence intervals

Suppose that one can construct a valid (1 − 𝛿 ) -CI for each of 𝐾 parameters of potential interest. If a data analyst uses an arbitrary data-dependent criterion to select some subset 𝑆 of

On the choice of the splitting ratio for the split likelihood ratio test

: The recently introduced framework of universal inference pro- vides a new approach to constructing hypothesis tests and confidence re-gions that are valid in finite samples and do not rely on any

Sequentially valid tests for forecast calibration

Forecasting and forecast evaluation are inherently sequential tasks. Predictions are often issued on a regular basis, such as every hour, day, or month, and their quality is monitored continuously.



On testing marginal versus conditional independence

We consider testing marginal independence versus conditional independence in a trivariate Gaussian setting. The two models are non-nested and their intersection is a union of two marginal

Probability inequalities for likelihood ratios and convergence rates of sieve MLEs

Let Y 1 ,...,Y n be independent identically distributed with density p 0 and let F be a space of densities. We show that the supremum of the likelihood ratios Π i=1 n p(Y i )/p 0 (Y i ), where the

Statistical Methods Related to the Law of the Iterated Logarithm

1. Extension and applications of an inequality of Ville and Wald. Let x 1… be a sequence of random variables with a specified joint probability distribution P. We shall give a method for obtaining

Confidence sequences for mean, variance, and median.

  • D. DarlingH. Robbins
  • Mathematics
    Proceedings of the National Academy of Sciences of the United States of America
  • 1967
UNIVERSITY OF CALIFORNIA, BERKELEY Communicated by J. Neyman, May 1, 1967 1. Normal Means and Variances.-Let x1,x2,... be independent, identically dis1 n tributed random variables and put ((t) =

Estimation of mixture models

A new information projection theory is derived to expand the approximating class to include its information closure, and the existence and uniqueness of a f* in the closure of the convex hull C is identified, such that D ( fpf* ) = infg∈CD fpg , where Dfpg is the Kullback-Leibler divergence.

Gaussian Mixture Clustering Using Relative Tests of Fit.

This work considers clustering based on significance tests for Gaussian Mixture Models (GMMs) based on SigClust method, and introduces a new test based on the idea of relative fit that test for whether a mixture of Gaussians provides a better fit relative to a single Gaussian.

A Polynomial Time Algorithm for Log-Concave Maximum Likelihood via Locally Exponential Families

The problem of computing the log-concave maximum likelihood distribution is formulated as a convex optimization problem, and solved via an approximate first-order method, and efficiently approximating the (sub) gradients of the objective function of this optimization problem is quite delicate, and is the main technical challenge in this work.

Sequential estimation of quantiles with applications to A/B testing and best-arm identification

Consider the problem of sequentially estimating quantiles of any distribution over a complete, fully-ordered set, based on a stream of i.i.d. observations. We propose new, theoretically sound and

Total positivity in structured binary distributions

It is proved that the maximum likelihood estimator (MLE) in an MTP2 binary exponential family exists if and only if the sign patterns $(1,-1) and $(-1,1) are represented in the sample for every pair of vertices; this implies that the MLE may exist with n=d samples, in stark contrast to unrestricted binary exponential families where $2^d$ samples are required.

Maximum Likelihood Estimation of the Distributions of Two Stochastically Ordered Random Variables

Abstract The maximum likelihood estimates and Ĝ of two distribution functions F and G are found, subject to the restrictions that for all x and that and Ĝ are of the discrete type. Random samples x