• Corpus ID: 231786611

Outlier-Robust Learning of Ising Models Under Dobrushin's Condition

@inproceedings{Diakonikolas2021OutlierRobustLO,
  title={Outlier-Robust Learning of Ising Models Under Dobrushin's Condition},
  author={Ilias Diakonikolas and Daniel M. Kane and Alistair Stewart and Yuxin Sun},
  booktitle={Annual Conference Computational Learning Theory},
  year={2021}
}
We study the problem of learning Ising models satisfying Dobrushin’s condition in the outlierrobust setting where a constant fraction of the samples are adversarially corrupted. Our main result is to provide the first computationally efficient robust learning algorithm for this problem with nearoptimal error guarantees. Our algorithm can be seen as a special case of an algorithm for robustly learning a distribution from a general exponential family. To prove its correctness for Ising models, we… 

Distributionally Robust Structure Learning for Discrete Pairwise Markov Networks

The primal minimax learning problem can be efficiently solved by leverag-ing sufficient statistics and greedy maximization in the ostensibly intractable dual formulation and derived near-optimal sample complexities match-ing existing results.

Trimmed Maximum Likelihood Estimation for Robust Learning in Generalized Linear Models

Under label corruptions, a classical heuristic called the iterative trimmed maximum likelihood estimator is proved to achieve minimax near-optimal risk on a wide range of generalized linear models, includ-ing Gaussian regression, Poisson regression and Binomial regression.

Chow-Liu++: Optimal Prediction-Centric Learning of Tree Ising Models

A new algorithm is introduced that carefully combines elements of the Chow-Liu algorithm with tree metric reconstruction methods to efficiently and optimally learn tree Ising models under a prediction-centric loss and is robust to model misspecification and adver-sarial corruptions.

Optimal SQ Lower Bounds for Robustly Learning Discrete Product Distributions and Ising Models

The optimal Statistical Query lower bounds for robustly learning certain families of discrete high-dimensional distributions are established, and a generic SQ lower bound is developed starting from low-dimensional moment matching constructions for discrete univariate distributions.

Streaming Algorithms for High-Dimensional Robust Statistics

The main result is for the task of high-dimensional robust mean estimation in (a strengthening of) Huber’s contamination model, which gives an efficient single-pass streaming algorithm with near-optimal error guarantees and space complexity nearly-linear in the dimension.

A Computationally Efficient Method for Learning Exponential Family Distributions

This work proposes a computationally efficient estimator that is consistent as well as asymptotically normal under mild conditions and shows that, at the population level, this method can be viewed as the maximum likelihood estimation of a re-parameterized distribution belonging to the same class of exponential family.

Learning and Covering Sums of Independent Random Variables with Unbounded Support

This work provides a set of simple conditions that allow the unbounded SIIRV to be learned with complexity poly(1/ǫ) bypassing the aforementioned lower bound and proves that any discrete unimodal exponential family with bounded constant-degree central moments can be approximated by a bounded subset of the initial (unbounded) parameter space.

Is Out-of-Distribution Detection Learnable?

This paper investigates the probably approximately correct (PAC) learning theory of OOD detection, which is proposed by researchers as an open problem and proves several impossibility theorems for the learnability of Ood detection under some scenarios.

Dimension-Free Rates for Natural Policy Gradient in Multi-Agent Reinforcement Learning

This work designs a scalable algorithm based on the Natural Policy Gradient framework that uses local information and only requires agents to communicate with neighbors within a certain range and converges to zero exponentially fast as a function of the range of communication.

References

SHOWING 1-10 OF 57 REFERENCES

On Learning Ising Models under Huber's Contamination Model

This work designs statistically optimal estimators that achieve an optimal dimension independent dependence on the fraction of corrupted data in the contaminated setting, while also simultaneously achieving high-probability error guarantees with optimal sample-complexity.

Robust Learning of Fixed-Structure Bayesian Networks

This work provides the first computationally efficient robust learning algorithm for this problem with dimension-independent error guarantees, which has near-optimal sample complexity, runs in polynomial time, and achieves error that scales nearly-linearly with the fraction of adversarially corrupted samples.

Robust Estimation of Tree Structured Ising Models

This paper focuses on the problem of robust estimation of tree-structured Ising models, and proves that this problem is unidentifiable, however, this unidentifiability is limited to a small equivalence class of trees formed by leaf nodes exchanging positions with their neighbors.

Learning Ising Models with Independent Failures

We give the first efficient algorithm for learning the structure of an Ising model that tolerates independent failures; that is, each entry of the observed sample is missing with some unknown

Robust Estimators in High Dimensions without the Computational Intractability

This work obtains the first computationally efficient algorithms for agnostically learning several fundamental classes of high-dimensional distributions: a single Gaussian, a product distribution on the hypercube, mixtures of two product distributions (under a natural balancedness condition), and k Gaussians with identical spherical covariances.

Robustly Learning any Clusterable Mixture of Gaussians

A new robust identifiability proof of clusters from a Gaussian mixture, which can be captured by the constant-degree Sum of Squares proof system, and a novel use of SoS-certifiable anti-concentration and a new characterization of pairs of Gaussians with small (dimension-independent) overlap in terms of their parameter distance.

Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

This work introduces a criterion, resilience, which allows properties of a dataset to be robustly computed, even in the presence of a large fraction of arbitrary additional data, and provides new information-theoretic results on robust distribution learning, robust estimation of stochastic block models, and robust mean estimation under bounded kth moments.

Agnostic Estimation of Mean and Covariance

This work presents polynomial-time algorithms to estimate the mean and covariance of a distribution from i.i.d. samples in the presence of a fraction of malicious noise with error guarantees in terms of information-theoretic lower bounds.

Settling the robust learnability of mixtures of Gaussians

This work gives the first provably robust algorithm for learning mixtures of any constant number of Gaussians, a new method for proving dimension-independent polynomial identifiability through applying a carefully chosen sequence of differential operations to certain generating functions.

Robustly learning mixtures of k arbitrary Gaussians

The main tools are an efficient partial clustering algorithm that relies on the sum-of-squares method, and a novel tensor decomposition algorithm that allows errors in both Frobenius norm and low-rank terms.
...