• Corpus ID: 219259999

Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators

@article{Kur2020ConvexRI,
  title={Convex Regression in Multidimensions: Suboptimality of Least Squares Estimators},
  author={Gil Kur and Fuchang Gao and Adityanand Guntuboyina and Bodhisattva Sen},
  journal={arXiv: Statistics Theory},
  year={2020}
}
The least squares estimator (LSE) is shown to be suboptimal in squared error loss in the usual nonparametric regression model with Gaussian errors for $d \geq 5$ for each of the following families of functions: (i) convex functions supported on a polytope (in fixed design), (ii) bounded convex functions supported on a polytope (in random design), and (iii) convex Lipschitz functions supported on any convex domain (in random design). For each of these families, the risk of the LSE is proved to… 
On Suboptimality of Least Squares with Application to Estimation of Convex Bodies
TLDR
It is established that Least Squares is mimimax sub-optimal, and achieves a rate of $\tilde{\Theta}_d (n-2/(d-1)})$ whereas the minimax rate is $\Theta_d(n^{-4/(d+3)})$.
High-dimensional asymptotics of likelihood ratio tests in the Gaussian sequence model under convex constraints
TLDR
These characterizations show that the power behavior of the LRT is in general non-uniform with respect to the Euclidean metric, and illustrate the conservative nature of existing minimax optimality and sub-optimality results for the L RT.
Adaptively Partitioning Max-Affine Estimators for Convex Regression
TLDR
A tractable convex piecewise-linear estimator which precomputes a partition of the training data by an adaptive version of farthest-point clustering, approximately hyperplanes over the partition cells by minimizing the regularized empirical risk, and projects the result into the max-affine class.
Noisy linear inverse problems under convex constraints: Exact risk asymptotics in high dimensions
In the standard Gaussian linear measurement model Y = Xμ0 + ξ ∈ Rm with a fixed noise level σ > 0, we consider the problem of estimating the unknown signal μ0 under a convex constraint μ0 ∈ K, where
Spectrahedral Regression
TLDR
Borders on how well spectrahedral functions can approximate arbitrary convex functions via statistical risk analysis are proved and an alternating minimization algorithm for the non-convex optimization problem of fitting the bestSpectrahedral function to a given data set is analyzed.
On the Minimal Error of Empirical Risk Minimization
We study the minimal error of the Empirical Risk Minimization (ERM) procedure in the task of regression, both in the random and the fixed design settings. Our sharp lower bounds shed light on the
SET STRUCTURED GLOBAL EMPIRICAL RISK MINIMIZERS ARE RATE OPTIMAL IN GENERAL DIMENSIONS
  • Han
  • Computer Science
  • 2020
TLDR
The present paper demonstrates that the global ERMs in such set structured models will indeed be rate-optimal, matching the lower bound even when the entropy integral diverges, and investigates models with set structures, which include image and edge estimation, binary classification, and multiple isotonic regression.
Minimax Rates for Conditional Density Estimation via Empirical Entropy
TLDR
This work considers the task of estimating a conditional density using i.i.d. samples from a joint distribution, and obtains matching (within logarithmic factors) upper and lower bounds on the minimax Kullback–Leibler risk in terms of the empirical Hellinger entropy for the conditional density class.
Rates of Estimation of Optimal Transport Maps using Plug-in Estimators via Barycentric Projections
Optimal transport maps between two probability distributions μ and ν on R have found extensive applications in both machine learning and statistics. In practice, these maps need to be estimated from
Statistics, computation, and adaptation in high dimensions
TLDR
Statistics, Computation, and Adaptation in High Dimensions: Foundations of a Models-based Approach to Models-Based Adaptation and Model-Based Models for Model-based Adaptation.

References

SHOWING 1-10 OF 52 REFERENCES
Isotonic regression in general dimensions
We study the least squares regression function estimator over the class of real-valued functions on $[0,1]^d$ that are increasing in each coordinate. For uniformly bounded signals and with a fixed,
Sharp oracle inequalities for Least Squares estimators in shape restricted regression
The performance of Least Squares (LS) estimators is studied in isotonic, unimodal and convex regression. Our results have the form of sharp oracle inequalities that account for the model
On Suboptimality of Least Squares with Application to Estimation of Convex Bodies
TLDR
It is established that Least Squares is mimimax sub-optimal, and achieves a rate of $\tilde{\Theta}_d (n-2/(d-1)})$ whereas the minimax rate is $\Theta_d(n^{-4/(d+3)})$.
Global risk bounds and adaptation in univariate convex regression
We consider the problem of nonparametric estimation of a convex regression function $$\phi _0$$ϕ0. We study the risk of the least squares estimator (LSE) under the natural squared error loss. We show
Optimality of Maximum Likelihood for Log-Concave Density Estimation and Bounded Convex Regression
TLDR
It is proved that estimating a log-concave density - even a uniform distribution on a convex set - up to a fixed accuracy requires the number of samples at least exponential in the dimension.
A Computational Framework for Multivariate Convex Regression and Its Variants
TLDR
A scalable algorithmic framework based on the augmented Lagrangian method to compute the nonparametric least squares estimator (LSE) of a multivariate convex regression function is proposed and a novel approach to obtain smooth convex approximations to the fitted (piecewise affine) convex LSE is developed.
A new perspective on least squares under convex constraint
TLDR
This paper presents three general results about the problem of estimating the mean of a Gaussian random vector, including an exact computation of the main term in the estimation error by relating it to expected maxima of Gaussian processes, a theorem showing that the least squares estimator is always admissible up to a universal constant in any problem of the above kind and a counterexample showing that least squares estimating may not always be minimax rate-optimal.
Multivariate Convex Regression at Scale
TLDR
This framework can solve instances of the convex regression problem with $n=10^5$ and $d=10$---a QP with 10 billion variables---within minutes; and offers significant computational gains compared to current algorithms.
An Improved Global Risk Bound in Concave Regression
TLDR
It is shown that indeed the logarithmic term is unnecessary and it is proved a risk bound which scales like n-4/5 up to constant factors and also extends to the case of model misspecification, where the true function may not be concave.
Convex Regression: Theory, Practice, and Applications
TLDR
The new excess risk upper bound is developed for the general empirical risk minimization setting without any shape constraints, and provides a probabilistic guarantee for cases with unbounded hypothesis classes, targets, and noise models.
...
...