Simple estimators of false discovery rates given as few as one or two p-values without strong parametric assumptions

@inproceedings{Bickel2013SimpleEO,
  title={Simple estimators of false discovery rates given as few as one or two p-values without strong parametric assumptions},
  author={David R. Bickel},
  booktitle={Statistical applications in genetics and molecular biology},
  year={2013}
}
  • D. Bickel
  • Published in
    Statistical applications in…
    22 June 2011
  • Mathematics
Abstract Multiple comparison procedures that control a family-wise error rate or false discovery rate provide an achieved error rate as the adjusted p-value or q-value for each hypothesis tested. However, since achieved error rates are not understood as probabilities that the null hypotheses are true, empirical Bayes methods have been employed to estimate such posterior probabilities, called local false discovery rates (LFDRs) to emphasize that their priors are unknown and of the frequency type… 

Figures and Tables from this paper

Small‐scale Inference: Empirical Bayes and Confidence Methods for as Few as a Single Comparison
TLDR
Simulations indicate that constrained LFDR MLEs perform markedly better than conventional methods, both in testing and in confidence intervals, for high values of the mixing proportion, but not for low values.
Estimators of the local false discovery rate designed for small numbers of tests
TLDR
Corrections of maximum likelihood estimators of the local false discovery rate (LFDR) of histogram-based empirical Bayes methods are introduced and it is found that HBE requires N to be at least 6-12 features to perform as well as the estimators proposed here, with the precise minimum N depending on p0 and dalt.
Empirical Bayes and fiducial effect-size estimation for small numbers of tests
Estimation of an effect size or other parameter of interest (POI), such as an average of differential abundance levels of metabolites or of the differential expression levels of genes, may be
Empirical Bayes Interval Estimates that are Conditionally Equal to Unadjusted Confidence Intervals or to Default Prior Credibility Intervals
  • D. Bickel
  • Mathematics
    Statistical applications in genetics and molecular biology
  • 2012
TLDR
Two first principles generate both classes of posteriors: a coherence principle and a relevance principle, which means effect size estimates given the truth of an alternative hypothesis cannot depend on whether that truth was known prior to observing the data or whether it was learned from the data.
Null Hypothesis Significance Testing Interpreted and Calibrated by Estimating Probabilities of Sign Errors: A Bayes-Frequentist Continuum
  • D. Bickel
  • Mathematics
    The American Statistician
  • 2020
Abstract Hypothesis tests are conducted not only to determine whether a null hypothesis (H0) is true but also to determine the direction or sign of an effect. A simple estimate of the posterior
Empirical Bayes estimation of posterior probabilities of enrichment: A comparative study of five estimators of the local false discovery rate
TLDR
For enrichment detection, the local FDR is recommended by estimating the LFDR by MLE given at least a medium number of GO terms, by BBE given a small number of Go terms, and by NMLE given either only 1 GO term or precise knowledge of Π0.
A simple yet efficient method of local false discovery rate estimation designed for genome-wide association data analysis
TLDR
This work uses the method of moments and introduces a simple, fast and efficient approach for LFDR estimation, assuming a chi-square model with one degree of freedom, which covers many situations in genome-wide association studies.
Blending Bayesian and frequentist methods according to the precision of prior information with applications to hypothesis testing
The proposed minimax procedure blends strict Bayesian methods with p values and confidence intervals or with default-prior methods. Two applications to hypothesis testing bring some implications to
A Simple Yet Efficient Parametric Method of Local False Discovery Rate Estimation Designed for Genome-Wide Association Data Analysis
TLDR
This work uses the method of moments and introduces a simple, fast and efficient method for LFDR estimation, which is evaluated by analyzing a comprehensive 1000 genomes-based genome-wide association data containing approximately 9.4 million single nucleotide polymorphisms.
...
...

References

SHOWING 1-10 OF 38 REFERENCES
Small-scale inference: Empirical Bayes and confidence methods for as few as a single comparison
TLDR
Simulations indicate that constrained LFDR MLEs perform markedly better than conventional methods, both in testing and in confidence intervals, for high values of the mixing proportion, but not for low values.
A Bayesian False Discovery Rate for Multiple Testing
Abstract Case-control studies of genetic polymorphisms and gene-environment interactions are reporting large numbers of statistically significant associations, many of which are likely to be
A direct approach to false discovery rates
TLDR
The calculation of the q‐value is discussed, the pFDR analogue of the p‐value, which eliminates the need to set the error rate beforehand as is traditionally done, and can yield an increase of over eight times in power compared with the Benjamini–Hochberg FDR method.
Empirical Bayes Interval Estimates that are Conditionally Equal to Unadjusted Confidence Intervals or to Default Prior Credibility Intervals
  • D. Bickel
  • Mathematics
    Statistical applications in genetics and molecular biology
  • 2012
TLDR
Two first principles generate both classes of posteriors: a coherence principle and a relevance principle, which means effect size estimates given the truth of an alternative hypothesis cannot depend on whether that truth was known prior to observing the data or whether it was learned from the data.
An empirical Bayes mixture method for effect size and false discovery rate estimation
TLDR
The proposed method outperforms existing effect size and false discovery rate estimation procedures in normal data simulations and nearly acheives the Bayes error for effect size estimation.
Estimating the Null Distribution to Adjust Observed Confidence Levels for Genome‐Scale Screening
TLDR
In a generic simulation study of genome-scale multiple testing, conditioning the observed confidence level on the estimated null distribution as an approximate ancillary statistic markedly improved conditional inference, indicating that estimation of the null distribution tends to exacerbate the conservative bias that results from modeling heavy-tailed data distributions with the normal family.
Empirical Bayes methods corrected for small numbers of tests
TLDR
Histogram-based empirical Bayes methods developed for analyzing data for large numbers of genes, SNPs, or other biological features tend to have large biases when applied to data with a smaller number of features, so corrections of maximum ∗ are introduced to analyze small-scale and mediumscale data.
Empirical bayes methods and false discovery rates for microarrays
TLDR
Two inferential approaches to this problem are discussed: an empirical Bayes method that requires very little a priori Bayesian modeling, and the frequentist method of “false discovery rates” proposed by Benjamini and Hochberg in 1995.
Correlated z-Values and the Accuracy of Large-Scale Statistical Estimates
  • B. Efron
  • Mathematics
    Journal of the American Statistical Association
  • 2010
TLDR
This paper concerns the accuracy of summary statistics for the collection of normal variates, such as their empirical cdf or a false discovery rate statistic, and shows that good accuracy approximations can be based on the root mean square correlation over all N ⋅ (N − 1)/2 pairs.
Large-Scale Inference: Empirical Bayes Methods for Estimation, Testing, and Prediction
TLDR
Technical aspects are not the focus of Principles of Applied Statistics, so this also explains why it does not dwell intently on nonparametric models.
...
...