Controlling the false discovery rate: a practical and powerful approach to multiple testing

  title={Controlling the false discovery rate: a practical and powerful approach to multiple testing},
  author={Yoav Benjamini and Yosef Hochberg},
  journal={Journal of the royal statistical society series b-methodological},
SUMMARY The common approach to the multiplicity problem calls for controlling the familywise error rate (FWER). This approach, though, has faults, and we point out a few. A different approach to problems of multiple significance testing is presented. It calls for controlling the expected proportion of falsely rejected hypotheses -the false discovery rate. This error rate is equivalent to the FWER when all hypotheses are true but is smaller otherwise. Therefore, in problems where the control of… 

Figures and Tables from this paper

Implementing false discovery rate control: increasing your power
Keeping the proportion of type I errors low among all significant results is a sensible, powerful, and easy-to-interpret way of addressing the multiple testing issue.
On the Adaptive Control of the False Discovery Rate in Multiple Testing With Independent Statistics
A new approach to problems of multiple significance testing was presented in Benjamini and Hochberg (1995), which calls for controlling the expected ratio of the number of erroneous rejections to the
Controlling the False Discovery Rate: A New Application to Account for Multiple and Dependent Tests in Local Statistics of Spatial Association
Assessing the significance of multiple and dependent comparisons is an important, and often ignored, issue that becomes more critical as the size of data sets increases. If not accounted for,
Comparison of Methods for Estimating the Number of True Null Hypotheses in Multiplicity Testing
Five methods used to estimate the number of true null hypotheses over a large number of hypotheses are compared and the lowest slope method, developed by Benjamini and Hochberg (2000), and the mean of differences method appear to perform the best.
Benjamini and Hochberg suggest that the false discovery rate may be the appropriate error rate to control in many applied multiple testing problems. A simple procedure was given there as an FDR
A direct approach to false discovery rates
The calculation of the q‐value is discussed, the pFDR analogue of the p‐value, which eliminates the need to set the error rate beforehand as is traditionally done, and can yield an increase of over eight times in power compared with the Benjamini–Hochberg FDR method.
Control of generalized error rates in multiple testing
Consider the problem of testing s hypotheses simultaneously. The usual approach restricts attention to procedures that control the probability of even one false rejection, the familywise error rate
Asymptotic results on adaptive false discovery rate controlling procedures based on kernel estimators
This paper proves that a class of plug-in procedures based on kernel estimators of the density of thedensity of thep-values are asymptotically more powerful in a setting where the hypotheses tested are independent, and shows that this increased asymPTotic power comes at the price of slower, non-parametric convergence rates for the FDP.


An improved Bonferroni procedure for multiple tests of significance
SUMMARY A modification of the Bonferroni procedure for testing multiple hypotheses is presented. The method, based on the ordered p-values of the individual tests, is less conservative than the
Statistical “Discoveries” and Effect-Size Estimation
Abstract Current methods of statistical inference are correct but incomplete. Small probability (α) of wrong null-hypothesis rejections can be misunderstood. Definitive rejections of null hypotheses,
A sequentially rejective test procedure based on a modified Bonferroni inequality
SUMMARY A sharper Bonferroni procedure for multiple tests of significance is derived. This procedure is an improvement of Hochberg's (1988) procedure which contrasts the individual P-values with
On the Optimality of Some Multiple Comparison Procedures
Abstract : Optimality criteria formulated in terms of the power functions of the individual tests are given for problems where several hypotheses are tested simultaneously. Subject to the constraint
A sharper Bonferroni procedure for multiple tests of significance
SUMMARY A simple procedure for multiple tests of significance based on individual p-values is derived. This simple procedure is sharper than Holm's (1979) sequentially rejective procedure. Both
A stagewise rejective multiple test procedure based on a modified Bonferroni test
SUMMARY Simes (1986) has proposed a modified Bonferroni procedure for the test of an overall hypothesis which is the combination of n individual hypotheses. In contrast to the classical Bonferroni
A Simple Sequentially Rejective Multiple Test Procedure
This paper presents a simple and widely ap- plicable multiple test procedure of the sequentially rejective type, i.e. hypotheses are rejected one at a tine until no further rejections can be done. It
Multiple Comparison Procedures: The Practical Solution
Abstract A practicing statistician looks at the multiple comparison controversy and related issues through the eyes of the users. The concept of consistency is introduced and discussed in relation to
More powerful procedures for multiple significance testing.
Some new, general and simple procedures are discussed and demonstrated by two examples from the medical literature: the neuropsychologic effects of unidentified childhood exposure to lead, and the sleep patterns of sober chronic alcoholics.
Impact of multiple comparisons in randomized clinical trials.