• Corpus ID: 232013535

Permutation-Based True Discovery Guarantee by Sum Tests

@inproceedings{Vesely2021PermutationBasedTD,
  title={Permutation-Based True Discovery Guarantee by Sum Tests},
  author={Anna Vesely and Livio Finos and Jelle J. Goeman},
  year={2021}
}
Sum-based global tests are highly popular in multiple hypothesis testing. In this paper we propose a general closed testing procedure for sum tests, which provides confidence lower bounds for the proportion of true discoveries (TDP), simultaneously over all subsets of hypotheses. Our method allows for an exploratory approach, as simultaneity ensures control of the TDP even when the subset of interest is selected post hoc. It adapts to the unknown joint distribution of the data through… 

Figures and Tables from this paper

Large-scale simultaneous inference under dependence
Simultaneous, post-hoc inference is desirable in large-scale hypotheses testing as it allows for exploration of data while deciding on criteria for proclaiming discoveries. It was recently proved
Faster Exact Permutation Testing: Using a Representative Subgroup
Non-parametric tests based on permutation, rotation or sign-flipping are examples of so-called group-invariance tests. These tests rely on invariance of the null distribution under a set of
The L\'evy combination test
A novel class of methods for combining p-values to perform aggregate hypothesis tests has emerged that exploit the properties of heavy-tailed Stable distributions. These methods offer important
Resampling-Based Multisplit Inference for High-Dimensional Regression
TLDR
A novel resampling-based method to construct an asymptotically exact test for any subset of hypotheses on coefficients in high-dimensional linear regression that requires less memory usage and shorter computation time, and can be scaled up to higher dimensions.
Smoothed nested testing on directed acyclic graphs
TLDR
It is proved that a broad class of smoothing strategies can be used with existing selection procedures to control the familywise error rate, false discovery exceedance rate, or false discovery rate, so long as the original test statistics are independent under the null.
Adaptive Cluster Thresholding with Spatial Activation Guarantees Using All-resolutions Inference
Classical cluster inference is hampered by the spatial specificity paradox. Given the null-hypothesis of no active voxels, the alternative hypothesis states that there is at least one active voxel in

References

SHOWING 1-10 OF 65 REFERENCES
Permutation-based simultaneous confidence bounds for the false discovery proportion
TLDR
In this paper, confidence upper bounds for the false discovery proportion are constructed, which are simultaneous over all rejection cut-offs, and a generalization of the method is provided that lets the user select the shape of the simultaneous confidence bounds; this gives the user more freedom in determining the power properties of the methods.
Large-scale simultaneous inference under dependence
Simultaneous, post-hoc inference is desirable in large-scale hypotheses testing as it allows for exploration of data while deciding on criteria for proclaiming discoveries. It was recently proved
Simultaneous control of all false discovery proportions in large-scale multiple hypothesis testing
Closed testing procedures are classically used for familywise error rate control, but they can also be used to obtain simultaneous confidence bounds for the false discovery proportion in all
Only closed testing procedures are admissible for controlling false discovery proportions
We consider the class of all multiple testing methods controlling tail probabilities of the false discovery proportion, either for one random set or simultaneously for many such sets. This class
The harmonic mean p-value for combining dependent tests
  • D. Wilson
  • Computer Science
    Proceedings of the National Academy of Sciences
  • 2019
TLDR
The harmonic mean p-value (HMP) is introduced, a simple to use and widely applicable alternative to Bonferroni correction motivated by Bayesian model averaging that greatly improves statistical power while maintaining control of the gold standard false positive rate.
Permutation-based true discovery proportions for fMRI cluster analysis.
TLDR
A user-friendly tool to find the percentage of true discoveries for each cluster while controlling the familywise error rate for multiple testing and taking into account that the cluster was chosen in a data-driven way is offered.
To permute or not to permute
TLDR
If the two distributions are not identical (as for example when they have unequal variances, correlations or skewness), then a permutation test for equality of means based on difference of sample means can have an inflated Type I error rate even when the means are equal.
Rank truncated product of P‐values, with application to genomewide association scans
TLDR
It is shown that, under realistic scenarios, forming the product of the K most significant P‐values provides increased power to detect genomewide association, while identifying a candidate set of good quality and fixed size for follow‐up studies.
Rotation tests
TLDR
This paper describes a generalised framework for doing Monte Carlo tests in multivariate linear regression and offers an exact Monte Carlo solution to a classical problem of multiple testing.
...
...