Corpus ID: 235458430

Statistical Query Lower Bounds for List-Decodable Linear Regression

@article{Diakonikolas2021StatisticalQL,
  title={Statistical Query Lower Bounds for List-Decodable Linear Regression},
  author={Ilias Diakonikolas and D. Kane and Ankit Pensia and Thanasis Pittas and Alistair Stewart},
  journal={ArXiv},
  year={2021},
  volume={abs/2106.09689}
}
We study the problem of list-decodable linear regression, where an adversary can corrupt a majority of the examples. Specifically, we are given a set T of labeled examples (x, y) ∈ R ×R and a parameter 0 < α < 1/2 such that an α-fraction of the points in T are i.i.d. samples from a linear regression model with Gaussian covariates, and the remaining (1−α)-fraction of the points are drawn from an arbitrary noise distribution. The goal is to output a small list of hypothesis vectors such that at… Expand

Tables from this paper

References

SHOWING 1-10 OF 70 REFERENCES
Clustering Mixture Models in Almost-Linear Time via List-Decodable Mean Estimation
TLDR
A novel and simpler near-linear time robust mean estimation algorithm in the α → 1 regime, based on a one-shot matrix multiplicative weightsinspired potential decrease is developed, providing a method to simultaneously cluster and downsample points using one-dimensional projections thus, bypassing the k-PCA subroutines required by prior algorithms. Expand
Statistical Query Lower Bounds for Robust Estimation of High-Dimensional Gaussians and Gaussian Mixtures
TLDR
A general technique that yields the first Statistical Query lower bounds for a range of fundamental high-dimensional learning problems involving Gaussian distributions is described, which implies that the computational complexity of learning GMMs is inherently exponential in the dimension of the latent space even though there is no such information-theoretic barrier. Expand
A General Characterization of the Statistical Query Complexity
TLDR
This work demonstrates that the complexity of solving general problems over distributions using SQ algorithms can be captured by a relatively simple notion of statistical dimension that is introduced, and is also the first to precisely characterize the necessary tolerance of queries. Expand
Statistical Query Algorithms for Mean Vector Estimation and Stochastic Convex Optimization
TLDR
This work studies the complexity of stochastic convex optimization given only statistical query access to the objective function, and derives nearly matching upper and lower bounds on the estimation (sample) complexity including linear optimization in the most general setting. Expand
Statistical Algorithms and a Lower Bound for Detecting Planted Cliques
TLDR
The main application is a nearly optimal lower bound on the complexity of any statistical query algorithm for detecting planted bipartite clique distributions when the planted clique has size O(n1/2 − δ) for any constant δ > 0. Expand
Statistical Query Algorithms and Low-Degree Tests Are Almost Equivalent
TLDR
This paper studies two of the most popular restricted computational models, the statistical query framework and low-degree polynomials, in the context of high-dimensional hypothesis testing, and finds that under mild conditions on the testing problem, the two classes of algorithms are essentially equivalent in power. Expand
Efficient noise-tolerant learning from statistical queries
TLDR
This paper formalizes a new but related model of learning from statistical queries, and demonstrates the generality of the statistical query model, showing that practically every class learnable in Valiant’s model and its variants can also be learned in the new model (and thus can be learning in the presence of noise). Expand
Learning from untrusted data
TLDR
An algorithm for robust learning in a very general stochastic optimization setting is provided that has immediate implications for robustly estimating the mean of distributions with bounded second moments, robustly learning mixtures of such distributions, and robustly finding planted partitions in random graphs. Expand
List-Decodable Mean Estimation in Nearly-PCA Time
TLDR
A new list-decodable mean estimation algorithm for bounded covariance distributions with optimal sample complexity and error rate, running in nearly-PCA time is proposed. Expand
List-decodable robust mean estimation and learning mixtures of spherical gaussians
TLDR
The problem of list-decodable (robust) Gaussian mean estimation and the related problem of learning mixtures of separated spherical Gaussians are studied and a set of techniques that yield new efficient algorithms with significantly improved guarantees are developed. Expand
...
1
2
3
4
5
...