Statistical and Computational Phase Transitions in Group Testing

  title={Statistical and Computational Phase Transitions in Group Testing},
  author={Amin Coja-Oghlan and Oliver Gebhard and Max Hahn-Klimroth and Alexander S. Wein and Ilias Zadik},
We study the group testing problem where the goal is to identify a set of k infected individuals carrying a rare disease within a population of size n , based on the outcomes of pooled tests which return positive whenever there is at least one infected individual in the tested group. We consider two different simple random procedures for assigning individuals to tests: the constant-column design and Bernoulli design . Our first set of results concerns the fundamental statistical limits. For the… 

Figures from this paper

The Franz-Parisi Criterion and Computational Trade-offs in High Dimensional Statistics

This paper formally connects a free-energy based criterion for hardness and formally establishes that for Gaussian additive models the “algebraic” notion of low-degree hardness implies failure of “geometric” local MCMC algorithms, and provides new low- degree lower bounds for sparse linear regression.

A second moment proof of the spread lemma

. This note concerns a well-known result which we term the “spread lemma,” which establishes the existence (with high probability) of a desired structure in a random set. The spread lemma was central

Average-Case Complexity of Tensor Decomposition for Low-Degree Polynomials

A model for random order-3 tensor decomposition where one component is slightly larger in norm than the rest (to break symmetry), and the components are drawn uniformly from the hypercube is considered.



Information-Theoretic and Algorithmic Thresholds for Group Testing

The sharp threshold for the number of tests required in this randomised design so that it is information-theoretically possible to infer the infection status of every individual is pinpointed.

Group testing and local search: is there a computational-statistical gap?

The fundamental limits of approximate recovery in the context of group testing is studied, and it is shown that a very simple local algorithm known as Glauber Dynamics does indeed succeed, and can be used to efficiently implement the well-known Smallest Satisfying Set (SSS) estimator.

Improved Bounds for Noisy Group Testing With Constant Tests per Item

This paper derives explicit algorithmic bounds for two commonly considered inference algorithms and thereby naturally extend the results of Scarlett & Cevher (2016) and Scarlett & Johnson (2020), providing improved performance guarantees for the efficient algorithms in these noisy group testing models.

On the All-or-Nothing Behavior of Bernoulli Group Testing

This article studies the problem of non-adaptive group testing, and shows that when too few tests are available, naively applying i.i.d. Bernoulli testing can lead to catastrophic failure, whereas “cutting one’s losses” and adopting a more carefully-chosen design can still succeed in attaining these less stringent criteria.

Group testing: an information theory perspective

This monograph surveys recent developments in the group testing problem from an information-theoretic perspective, and identifies several regimes where existing algorithms are provably optimal or near-optimal, as well as regimes where there remains greater potential for improvement.

Phase Transitions in Group Testing

In the noiseless case with the number of defective items k scaling with the total number of items p as O(pθ) (θ ∈ (0, 1), it is shown that the probability of reconstruction error tends to one when n ≤ k log2 p/k (1 + o(1)), but vanishes when n ≥ c(θ), thus providing an exact threshold on the required number measurements.

Group Testing Algorithms: Bounds and Simulations

It is shown that DD outperforms COMP, that DD is essentially optimal in regimes where K ≥ √N, and that no algorithm can perform as well as the best nonrandom adaptive algorithms when K > N0.35.

Notes on Computational Hardness of Hypothesis Testing: Predictions using the Low-Degree Likelihood Ratio

These notes survey and explore an emerging method, which is called the low-degree method, for predicting and understanding statistical-versus-computational tradeoffs in high-dimensional inference problems, which posits that a certain quantity gives insight into how much computational time is required to solve a given hypothesis testing problem.

Informative Dorfman Screening

This article uses individuals’ risk probabilities to formulate new informative decoding algorithms that implement Dorfman retesting in a heterogeneous population, and introduces the concept of “thresholding” to classify individuals as “high” or “low risk,” so that separate, risk‐specific algorithms may be used, while simultaneously identifying pool sizes that minimize the expected number of tests.

Improved group testing rates with constant column weight designs

This work shows that the rate of the practical COMP detection algorithm is increased by 31% in all sparsity regimes, and gives an algorithm-independent upper bound for the constant column weight case.