• Corpus ID: 219177383

Universal Robust Regression via Maximum Mean Discrepancy

  title={Universal Robust Regression via Maximum Mean Discrepancy},
  author={Pierre Alquier and Mathieu Gerber},
  journal={arXiv: Statistics Theory},
Many datasets are collected automatically, and are thus easily contaminated by outliers. In order to overcome this issue, there was recently a regain of interest in robust estimation methods. However, most of these methods are designed for a specific purpose, such as estimation of the mean, or linear regression. We propose estimators based on Maximum Mean Discrepancy (MMD) optimization as a universal framework for robust regression. We provide non-asymptotic error bounds, and show that our… 
2 Citations

Figures from this paper

Discrepancy-based Inference for Intractable Generative Models using Quasi-Monte Carlo
Intractable generative models are models for which the likelihood is unavailable but sampling is possible. Most approaches to parameter inference in this setting require the computation of some
Estimation of copulas via Maximum Mean Discrepancy
This paper deals with robust inference for parametric copula models. Estimation using Canonical Maximum Likelihood might be unstable, especially in the presence of outliers. We propose to use a


We study robust estimators of the mean of a probability measure P, called robust empirical mean estimators. This elementary construction is then used to revisit a problem of aggregation and a problem
Robust linear least squares regression
We consider the problem of robustly predicting as well as the best linear combination of d given functions in least squares regression, and variants of this problem including constraints on the
Finite sample properties of parametric MMD estimation: robustness to misspecification and dependence
This paper tackles the problem of universal estimation using a minimum distance estimator presented in Briol et al. (2019) based on the Maximum Mean Discrepancy, and shows that the estimator is robust to both dependence and to the presence of outliers in the dataset.
Distribution-robust mean estimation via smoothed random perturbations
We consider the problem of mean estimation assuming only finite variance. We study a new class of mean estimators constructed by integrating over random noise applied to a soft-truncated empirical
Geometric median and robust estimation in Banach spaces
In many real-world applications, collected data are contaminated by noise with heavy-tailed distribution and might contain outliers of large magnitude. In this situation, it is necessary to apply
MMD-Bayes: Robust Bayesian Estimation via Maximum Mean Discrepancy
A pseudo-likelihood based on the Maximum Mean Discrepancy, defined via an embedding of probability distributions into a reproducing kernel Hilbert space is built, and it is shown that this MMD-Bayes posterior is consistent and robust to model misspecification.
MONK - Outlier-Robust Mean Embedding Estimation by Median-of-Means
This paper shows how the recently emerged principle of median-of-means can be used to design estimators for kernel mean embedding and MMD with excessive resistance properties to outliers, and optimal sub-Gaussian deviation bounds under mild assumptions.
Robust high dimensional learning for Lipschitz and convex losses
We establish risk bounds for Regularized Empirical Risk Minimizers (RERM) when the loss is Lipschitz and convex and the regularization function is a norm. We obtain these results in the i.i.d. setup
Robust classification via MOM minimization
We present an extension of Chervonenkis and Vapnik’s classical empirical risk minimization (ERM) where the empirical risk is replaced by a median-of-means (MOM) estimator of the risk. The resulting
Loss Minimization and Parameter Estimation with Heavy Tails
The technique can be used for approximate minimization of smooth and strongly convex losses, and specifically for least squares linear regression and low-rank covariance matrix estimation with similar allowances on the noise and covariate distributions.